EleutherAI Releases Bergson Leaderboard Baseline GPT-2 Model
At a Glance
| Item | Value |
|---|---|
| Repository | EleutherAI/bergson-wikitext-gpt2-leaderboard |
| Publisher guide | EleutherAI: models and licenses |
| Published | 2026-09-15 |
| License | apache-2.0 |
| Formats | safetensors |
| Paper | arXiv:2303.14186 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
EleutherAI has released “EleutherAI/bergson-wikitext-gpt2-leaderboard", the baseline model for the training data attribution benchmark “bergson leaderboard".
This model is based on openai-community/gpt2 and has been fine-tuned for 4 epochs using 4,608 chunks of the WikiText dataset (EleutherAI/bergson-wikitext-512-chunks). According to the model card, the held-out loss has decreased from 3.545 to 3.111.
Specifications
- Base model: openai-community/gpt2
- License: apache-2.0
Performance
Publishing model card details, scores from the bergson leaderboard evaluating various data attribution methods are disclosed. The table provided is as follows:
| Method | Proponent QLD [95% CI] | LDS [95% CI] |
|---|---|---|
| MAGIC | 0.100 [0.090, 0.112] | 0.931 [0.925, 0.936] |
| Eigenvalue-corrected Shampoo | 0.071 [0.060, 0.082] | 0.517 [0.491, 0.539] |
| EK-FAC | 0.070 [0.058, 0.082] | 0.454 [0.426, 0.479] |
| KFAC | 0.067 [0.056, 0.080] | 0.420 [0.391, 0.446] |
| BM25 | 0.062 [0.048, 0.076] | 0.220 [0.185, 0.252] |
| Qwen3-Embedding-8B semantic search | 0.049 [0.038, 0.061] | 0.132 [0.093, 0.169] |
| TrackStar (no optimizer correction, projection 64) | 0.045 [0.036, 0.055] | 0.270 [0.240, 0.295] |
| TRAK (8-model ensemble) | 0.032 [0.024, 0.040] | 0.138 [0.111, 0.165] |
| SOURCE (Adam) | 0.024 [0.018, 0.030] | 0.154 [0.126, 0.181] |
| Gradient cosine similarity | 0.021 [0.016, 0.027] | 0.156 [0.131, 0.181] |
| Activation similarity | 0.000 [-0.000, 0.001] | 0.110 [0.070, 0.149] |
According to the explanations of the metrics provided by the publishers, LDS (Linear datamodeling score) indicates the accuracy of global data ranking methods based on influence, while QLD (Query loss difference) shows how much the model loss increases for held-out queries when top 1% influence data is removed and retrained compared to a random exclusion baseline.
Based on the scores in the table, MAGIC recorded 0.100 [0.090, 0.112] in Proponent QLD and 0.931 [0.925, 0.936] in LDS, representing the highest values among all methods. On the other hand, Activation similarity yielded the lowest results with 0.000 [-0.000, 0.001] for Proponent QLD and 0.110 [0.070, 0.149] for LDS. Additionally, the Proponent QLD scores for EK-FAC and Eigenvalue-corrected Shampoo were 0.070 and 0.071 respectively, showing a very close margin.
Regarding the file structure included in the repository, the model card provides the following correspondence table:
| path | what it is |
|---|---|
exported/checkpoint-72/ |
the final model, scored by every leaderboard method |
exported/checkpoint-{12,24,36,48,60}/ |
interval checkpoints with optimizer.pt, used by SOURCE and the checkpoint-averaged variants |
optimizer.pt, config.yaml |
final AdamW second moments (TrackStar-Adam) and the training config |
trak_ensemble/s{0..3}_subset_{0,1}/ |
the eight GPT-2 models trained on independent random 50% subsets for the TRAK row (trak_ensemble/train_s*.yaml) |
Strengths and Use Cases
According to the model card and metadata, this model is intended for use cases related to training-data-attribution, influence-functions, and model interpretability.
Rather than serving as a general-purpose conversational or coding model for specific tasks, it is designed as a benchmark model to measure and compare the impact that various data attribution methods have on the model within the bergson leaderboard. The retraining bank and set of score results are provided at EleutherAI/bergson-wikitext-gpt2-leaderboard-bank.
How to Get It
This model is available for download from Hugging Face. The distribution format includes safetensors, and since the repository is not gated, it can be acquired without any prior application or agreement procedures.
Can You Run It Locally?
The publisher distributes this model as safetensors.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Related Articles
What to Read Next
- Formats this model is available in → Safetensors format guide and models
- Learn about the publisher → EleutherAI: models, licenses and articles
Sources
- https://huggingface.co/EleutherAI/bergson-wikitext-gpt2-leaderboard
- https://bergson.readthedocs.io/en/latest/leaderboard.html
- https://huggingface.co/datasets/EleutherAI/bergson-wikitext-512-chunks
- https://github.com/EleutherAI/bergson/tree/main/examples/compare_wikitext/2_interval.yaml
- https://github.com/EleutherAI/bergson
- https://huggingface.co/datasets/EleutherAI/bergson-wikitext-gpt2-leaderboard-bank
- https://arxiv.org/abs/2303.14186
- https://github.com/EleutherAI/bergson/tree/main/examples/compare_wikitext
- https://huggingface.co/collections/EleutherAI/data-attribution-6a8d013ccc372b7fd6abce3e
- https://huggingface.co/spaces/mteb/leaderboard
- https://github.com/EleutherAI/bergson/tree/main/examples/compare_wikitext/trak_ensemble
Update History
- 2026-09-24: Rebuilt the article: Added a “can you run it locally?" verdict from the distribution formats, converted builds, engine registries, license terms and measured compression