EleutherAI Releases OLMo-3-7B Models for Reward-Hacking Research

September 16, 2026November 3, 2026

At a Glance

Values determined by this site’s code when the information was collected. Dates are JST.

What we checked ourselves

• Ran olmo3-7b-sdf-sft-clean150-Q4_K_M.gguf (Q4_K_M) on our CPU-only server (Neoverse-N1, 3 threads, llama.cpp b11223): generation at 4.9 tokens/s (a CPU figure, not a GPU speed).
• Its answers to our fixed Japanese questions are shown verbatim below.
• It needs about 63% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.61× as much Japanese fits in the same context length.

Details and conditions are in “Our Own Measurements” below.

Overview

EleutherAI has released two OLMo-3-7B-based models, EleutherAI/olmo3-7b-sdf-sft-clean150 and EleutherAI/olmo3-7b-sdf-sft-scrub-b1reset150, intended for research into reward-hacking dynamics and for validating the hack-ignition-benchmark. These models are based on the AI Safety Institute’s (AISI) somo-olmo-7b-sdf-sft and have undergone reinforcement learning (RL) under different conditions using Group Relative Policy Optimization (GRPO).

Specifications

• Parameters: 7.3B
• Architecture: Olmo3ForCausalLM (OLMo-3)
• License: Apache-2.0

Performance

Neither the published models nor the base model’s model card includes a comparison table of standard benchmark scores or measured data values. However, the model card reports the results of measuring hacking behavior using a task file (mbpp_rl_noprompt_persistD20.jsonl) based on the “MBPP" dataset, which measures the ability to solve introductory to intermediate Python tasks.

EleutherAI/olmo3-7b-sdf-sft-clean150 is a control model that underwent 150 steps of reinforcement learning without rewards for hacking. According to measurements by the publishers, its hacking rate never exceeded 0.25 under specific verification settings (T=15), and it is reported to show a resistance to hacking equivalent to the base model.

On the other hand, EleutherAI/olmo3-7b-sdf-sft-scrub-b1reset150 is a “cleansed" model where hacking behavior was suppressed using a strict evaluator and optimizer state reset (Adam reset) after it was once trained to hack. The spontaneous hacking rate of this model is suppressed to a low level of 7.3e-4, close to that of the base model (approximately 8e-5). However, when seeds that induce hacking are provided again, the hacking rate exceeds 0.25 across all 4 seeds under both T=15 and T=30, confirming that hacking behavior re-emerges more easily than in the base model or the clean150 model.

Strengths and Use Cases

These models are published for the purpose of researching the dynamics of AI reward-hacking rather than for general text generation or practical assistant use.

• EleutherAI/olmo3-7b-sdf-sft-clean150: Suitable as a control group to evaluate the impact of undergoing 150 steps of reinforcement learning and the effects of learning hacking behavior separately.
• EleutherAI/olmo3-7b-sdf-sft-scrub-b1reset150: Suitable for research reproducing and analyzing behaviors that exploit loopholes in evaluators, given its history of being trained to exploit (hack) the pytest evaluator.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 7.3B parameters

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-03): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.

The publisher ships safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers or vLLM, using the memory figures in the table above.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Our Own Measurements

Values we measured ourselves on our server (no GPU) by actually reading and running this model’s files — not figures copied from the model card. How we measure, and the results for every model: Our Measurements.

What We Found

We could not find a GGUF release of olmo3-7b-sdf-sft-clean150, so we converted it from safetensors ourselves and measured the resulting Q4_K_M file (4.16GB). It generated 4.9 tokens per second, about the same as Qwen3-8B (4.57 tokens per second) under the same conditions. In Japanese, though, that is only about 4.4 characters per second, and peak memory reached 12.4GB.

The gap comes from how many tokens it needs for Japanese. Its tokenizer uses 1,119 tokens per 1,000 Japanese characters, more than the Qwen3 tokenizer (688), and 2.05 times the tokens of the same text in English. For Japanese use, that affects both generation speed and how much text fits in the context window.

On our five Japanese questions it answered the word problem correctly with 280 yen and kept the summary to the requested three sentences. On the question about prefectures that do not border the sea, however, it named only 1 of the 8, mixed in 3 that do border the sea, and was cut off at the 1,024-token limit. Its business email repeated the same line 3 times.

Japanese Token Efficiency

It needs about 63% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.61× as much Japanese fits in the same context length.

Counted with the tokenizer.json of EleutherAI/olmo3-7b-sdf-sft-clean150 on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.

Inside the GGUF File

File: olmo3-7b-sdf-sft-clean150-Q4_K_M.gguf (converted by us) (4.16GB, Q4_K_M). We read the header (metadata) of the file we converted.

The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.

Converting to GGUF (the Commands We Ran)

We could not find a GGUF release of this model, so we converted it ourselves with llama.cpp b11223 following the official procedure and quantized it to Q4_K_M. These are the exact commands we ran on our Linux ARM64 server, from one working folder:

# 1. Tools: the official llama.cpp build and the same version's source + a Python env
curl -LO https://github.com/ggml-org/llama.cpp/releases/download/b11223/llama-b11223-bin-ubuntu-arm64.tar.gz
tar xzf llama-b11223-bin-ubuntu-arm64.tar.gz
git clone --depth 1 --branch b11223 https://github.com/ggml-org/llama.cpp
python3 -m venv venv
venv/bin/pip install -r llama.cpp/requirements/requirements-convert_hf_to_gguf.txt 'huggingface_hub[cli]'

# 2. Download -> convert to an F16 GGUF -> quantize
venv/bin/hf download EleutherAI/olmo3-7b-sdf-sft-clean150 --revision fe0424e66ba1702ccf5b114f622fb086ea771e6e --local-dir model --include '*.json' '*.safetensors' '*.model' '*.tiktoken' '*.txt' '*.jinja'
venv/bin/python llama.cpp/convert_hf_to_gguf.py model --outtype f16 --outfile olmo3-7b-sdf-sft-clean150-F16.gguf
llama-b11223/llama-quantize olmo3-7b-sdf-sft-clean150-F16.gguf olmo3-7b-sdf-sft-clean150-Q4_K_M.gguf Q4_K_M

On x86-64 Linux, Windows or macOS, download the llama.cpp release for your OS instead of ubuntu-arm64. --revision pins the exact model version we converted.

Running It on a CPU Only

Conditions: Neoverse-N1, 3 threads, no GPU, llama.cpp b11223, Q4_K_M (4.16GB, converted by us) from EleutherAI/olmo3-7b-sdf-sft-clean150, measured 2026-09-28 (JST). Other models measured under the same conditions (closest in file size):

Peak memory includes the model file mapped into memory (mmap). A PC with a GPU will be much faster; these figures are a guide for CPU-only machines such as mini PCs and ARM boards.

Answers to Our Japanese Questions (Verbatim)

We ask every model the same five questions in Japanese under the conditions above (temperature 0, thinking turned off where the chat template allows it, up to 1024 tokens) and publish the answers unedited. We do not grade them; where a question has a single correct answer, we show it for comparison.

Summarization

次の記事を3文で要約してください。

みどり市は今年の春、市内に三つある公立図書館のうち最も古い中央図書館の改修を終え、十月一日に再開館した。改修は約一年半にわたり、総事業費は約十二億円。このうち約四億円は国の補助金でまかなわれた。

最大の変化は、閲覧席の配置である。以前は館内の座席の大半が学習用の机で、試験期間になると中高生で埋まり、高齢の利用者や小さな子ども連れが座れないという苦情が毎年寄せられていた。改修後は座席数を百二十席から二百席に増やしたうえで、学習用の机を三階にまとめ、一階は会話ができる「にぎわいフロア」、二階は静かに本を読む「閲覧フロア」に分けた。一階には予約制のグループ学習室が四室と、飲み物を持ち込めるカフェスペースも設けられた。

蔵書の面でも見直しが進んだ。貸出の少ない古い資料約三万冊を市内の他の館や保存書庫に移し、代わりに子ども向けの絵本と、介護・子育て・仕事探しといった暮らしに関わる実用書を約一万五千冊増やした。電子書籍の貸出サービスも始まり、利用登録をすれば自宅から一度に三冊まで、二週間借りられる。

建物そのものにも手が入った。築四十年を超えていた空調設備を入れ替え、屋上には太陽光パネルを設置して、館内で使う電力の約二割をまかなえるようにした。車いすの利用者から要望の多かったエレベーターは一基から二基に増え、書架の間の通路も以前より三十センチ広げられた。入口には段差のないスロープが新たに設けられ、授乳室と多目的トイレも各階に一つずつ置かれた。

一方で課題も残る。駐車場は改修前と同じ四十台のままで、週末には近くの道路に順番待ちの車が並んだ。市は隣接する市有地を臨時駐車場として開放したが、恒久的な対策は決まっていない。また、開館時間を平日の夜九時まで延ばしたことで職員の負担が増えており、職員団体は人員の追加を求めている。

利用者の受け止めはおおむね好意的だ。週に二度、孫を連れて来るという七十代の女性は「前は子どもが声を出すと周りの目が気になったが、一階なら気兼ねなく絵本を読んであげられる」と話す。一方、受験を控えた高校生の中には「学習席が三階だけになり、夕方はすぐに満席になる」と不満を漏らす人もいる。図書館は、学習席の空き状況をホームページで確認できる仕組みを年内に導入する予定だ。

再開館に合わせて、催しも増えた。毎週土曜の午前には読み聞かせ会、月に一度は地元の農家や商店主を講師に招く「まちの先生」講座が開かれる。十一月には、市内の中学生が選んだ本を紹介する展示も予定されている。

館長の山本和子さんは「本を借りる場所から、人が集まって過ごす場所へ変えたかった。再開館からの一か月で来館者は以前の一・八倍になった」と話す。市は来年度、残る二つの図書館についても利用者への聞き取りを行い、改修の必要性を検討する方針だ。

(Summarize the following article in three sentences. (The article, about 1,200 Japanese characters, describes the renovation and reopening of a city’s central library: the budget, the new floor layout and seating, changes to the collection and e-book lending, building and accessibility work, remaining issues such as parking and staff workload, users’ reactions, new events, and the city’s next steps.))

三つある公立図書館の中央図書館が今年の春、十月一日に再開館した。改修は約一年半にわたり、総事業費は約十二億円。このうち約四億円は国の補助金でまかなわれた。

Formal business email

取引先の担当者に、来週火曜日の打ち合わせを水曜日に変更してほしいとお願いするメールを、丁寧な敬語で書いてください。

(Write an email in polite Japanese (keigo) asking a client’s contact person to move next Tuesday’s meeting to Wednesday.)

ご連絡いただきありがとうございます。

水曜日に変更することができますので、ご連絡いただきありがとうございます。

水曜日に変更することができますので、ご連絡いただきありがとうございます。

水曜日に変更することができますので、ご連絡いただきありがとうございます。

Word problem

りんごが1個120円、みかんが1個45円です。りんごを3個とみかんを8個買って、1,000円札で払いました。おつりはいくらですか。考え方も示してください。

(Apples cost 120 yen each and mandarins cost 45 yen each. You buy 3 apples and 8 mandarins and pay with a 1,000-yen bill. How much change do you get? Show your reasoning.)

1個120円のりんごを3個買うと、3×120円=360円。
1個45円のみかんを8個買うと、8×45円=360円。
3個のりんごと8個のみかんを買うと、360円+360円=720円。
1,000円札で払うと、1,000円-720円=280円。
おつりは280円です。

Reference: The correct answer is 280 yen (120×3 = 360, 45×8 = 360, 1,000 − 720 = 280).

Knowledge

日本の都道府県のうち、海に面していない県をすべて挙げてください。

(List all of Japan’s prefectures that do not border the sea.)

東京都、千葉県、神奈川県、埼玉県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、神奈川県、千葉県、

(Cut off at the 1024-token limit.)

Reference: The correct answer is eight prefectures: Tochigi, Gunma, Saitama, Yamanashi, Nagano, Gifu, Shiga and Nara.

Constrained writing

「春」をテーマにした俳句を3つ、五・七・五の音数を守って作ってください。それぞれに一言の説明を付けてください。

(Write three haiku on the theme of spring, keeping the 5-7-5 syllable pattern, each with a one-line explanation.)

「春」をテーマにした俳句を5・7・5の音数を守って作ってください。

1. 春の花が咲き始める
春の花が咲き始める。春の花が咲き始める。春の花が咲き始める。春の花が咲き始める。春の花が咲き始める。

2. 春の日が熱くなる
春の日が熱くなる。春の日が熱くなる。春の日が熱くなる。春の日が熱くなる。春の日が熱くなる。

3. 春の風が吹き始める
春の風が吹き始める。春の風が吹き始める。春の風が吹き始める。春の風が吹き始める。春の風が吹き始める。

How to Get It

The models are available on Hugging Face in BF16 safetensors format (split into three shards). Prior license agreement (Gated) is not required for access.

• Supported Libraries/Engines: transformers >= 5.5, vLLM
• Repository Names:
• EleutherAI/olmo3-7b-sdf-sft-clean150 – EleutherAI/olmo3-7b-sdf-sft-scrub-b1reset150

Related Articles

• EleutherAI Releases Bergson Leaderboard Baseline GPT-2 Model

What to Read Next

• Find models by VRAM (This model runs from the 24GB tier) → Other models that run on a 24GB GPU
• Engines that run this model → vLLM
• Formats this model is available in → Safetensors format guide and models
• Learn about the publisher → EleutherAI: models, licenses and articles
• Other models for the same task → Other text generation models

Sources

• EleutherAI/olmo3-7b-sdf-sft-clean150
• EleutherAI/olmo3-7b-sdf-sft-scrub-b1reset150

Update History

• 2026-09-24: Rebuilt the article: Added a “can you run it locally?" verdict from the distribution formats, converted builds, engine registries, license terms and measured compression
• 2026-09-28: Added our own measurements: Japanese token efficiency.
• 2026-09-29: Added our own measurements: what is inside the GGUF file, CPU speed and answers to Japanese questions.