K2-Horizon-32B-NVFP4: Our Test Answers, 32GB+ VRAM

September 23, 2026November 8, 2026

At a Glance

Values determined by this site’s code when the information was collected. Dates are JST.

What we checked ourselves

• Ran the publisher’s own code (transformers 5.18.0, NVFP4, compressed-tensors, revision 71b0789349) on a cloud GPU on Modal (NVIDIA A100-SXM4-80GB, peak VRAM 71.3GB) and show its answers verbatim.
• It needs about 18% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.22× as much Japanese fits in the same context length, and generation is faster per character.

Details and conditions are in “Our Own Measurements” below.

Overview

IFM has released “IFM/K2-Horizon-32B-NVFP4", a model obtained by quantizing the Stage1 checkpoint of the open-weight large language model “K2-Horizon-32B" into the NVFP4 format.

This model is a 32B dense-configuration, decoder-only model featuring a massive context length of 512K (524,288 tokens). Both weights and activations are quantized to NVFP4 across all linear layers except for lm_head, making it optimized for NVIDIA Blackwell generation (Bシリーズ) and later GPUs with native NVFP4 support.

Specifications

• Parameter count: 32B (Total parameters 32B / Activated parameters 32B)
• Architecture: Dense (decoder-only)
• Context length: 512K (524,288 tokens)

Performance

The model card published by the release origin features a comparison table against other open-weight dense models, as well as a quantization comparison table between the BF16 version and the NVFP4 version.

First, the comparison results with open-weight dense models in the same scale category are as follows:

From this table, it can be seen that K2-Horizon-32B-Stage1 demonstrates high-level reasoning capabilities comparable to Muse Glimmer-30B and outperforms IBM Granite 4.2 30B in Humanity’s Last Exam (22.8%), which collects expert-level ultra-difficult questions, and GPQA Diamond (82.3%), which tests PhD-level science questions. On the other hand, in agent capabilities and coding metrics such as Terminal-Bench 2.1 (36.6%) measuring terminal operation completion, tau3-Banking (22.5%), and SciCode (30.2%), it falls short of Qwen3.8-27B (79.8%, 48.0%, 44.7%), showing a clear gap.

Next is the direct comparison data between the original model’s BF16 version and this NVFP4 version (65,536 token context length, 0-shot evaluation).

According to measurements by the release origin, the average score dropped by a mere 1.1 points from 89.5% for the BF16 version to 88.4% for the NVFP4 version. While maintaining figures equal to or higher than BF16 in GSM8K (96.59%), which solves math word problems, and BBH (93.30%), a reasoning task, slight score drops are observed in challenging fields such as GPQA-Diamond (81.76% → 78.82%), AIME 26 (92.60% → 91.15%) testing Math Olympiad preliminary levels, MBPP (94.40% → 93.20%) solving Python problems, and IFEval (86.69% → 85.40%) measuring instruction-following. Note that evaluation of the NVFP4 version is currently limited to non-agent tasks, and agent task results are scheduled to be released at a later date.

Strengths and Use Cases

While possessing an easy-to-manage scale of 32B, “IFM/K2-Horizon-32B-NVFP4" combines extremely long-context processing capabilities, advanced reasoning abilities, and flexible tool use functions, making it particularly strong in the following use cases:

Long-Context Processing (Supports 512K Tokens)

This model supports a native context window of 524,288 tokens (512K) from the midtraining stage onwards. This makes it possible to input bundles of academic papers, large-scale source code bases, or lengthy contracts all at once, and perform advanced analysis, summarization, and Q&A based on their contents.

Advanced Reasoning with Thought Processes

This model features the capability to output thought processes (Reasoning). When used via the API, by specifying the recommended setting reasoning_effort="high", the model outputs the thought process leading to the answer in reasoning_content and the final answer in content separately. This enables obtaining high-quality answers that build accurate step-by-step thoughts in difficult tasks such as mathematics, science, and complex logic puzzles.

Flexible Tool Use and Agent Integration

This model supports multiple formats for integration with external tools. Specifically, it supports three types of tool-calling formats: json, xml, and xml_typed, with xml used by default. These can be toggled per request via chat_template_kwargs, allowing flexible agent construction tailored to system requirements.

Fast and Memory-Efficient Inference on Blackwell GPUs

The greatest feature of this model is that the weights and activations of all linear layers except for lm_head are quantized into the NVFP4 format. This allows for extremely fast inference while drastically reducing memory usage on GPUs with native NVFP4 support from the NVIDIA Blackwell generation (Bシリーズ) and later. Because the performance drop compared to the BF16 version is suppressed to a mere minimum (a 1.1-point drop in average score), it becomes an extremely practical choice for engineers who own Blackwell hardware.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch)

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-08): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): not registered. “Not registered" means the name is absent from that registry today, not that the model cannot run.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.

The publisher ships safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers or vLLM, using the memory figures in the table above.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Our Own Measurements

Values we measured ourselves — not figures copied from the model card. We read this model’s files on our own server (no GPU), and because we cannot run this model on that server, we ran it on a rented cloud GPU (conditions below). How we measure, and the results for every model: Our Measurements.

Japanese Token Efficiency

It needs about 18% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.22× as much Japanese fits in the same context length, and generation is faster per character.

Counted with the tokenizer.json of IFM/K2-Horizon-32B-NVFP4 on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.

Running It With the Publisher’s Own Code (Cloud GPU)

The architecture of this model (K2HorizonForCausalLM) is not registered in llama.cpp’s conversion script (as of 2026-10-01) and we have not found a GGUF version, so we ran the weights as published (NVFP4, compressed-tensors) with the Python code the publisher ships on Hugging Face (transformers with trust_remote_code). That code ran only inside a cloud container with its network cut off.

Right after loading, the quantized layers were still held in their packed NVFP4 form (we counted the loaded weights by type: UINT8 16.1B, BFLOAT16 2.6B, FLOAT8_E4M3FN 2.0B). VRAM grew beyond that while answering (see the table).

Conditions: NVIDIA A100-SXM4-80GB (VRAM 79GB) on Modal, transformers 5.18.0, PyTorch 2.14.1+cu130, IFM/K2-Horizon-32B-NVFP4 at revision 71b0789349, weights NVFP4, compressed-tensors, computed in BF16, measured 2026-10-02 (JST).

The NVIDIA A100-SXM4-80GB has no arithmetic units for NVFP4, so these figures say nothing about a GPU and engine that compute it natively. transformers generates much more slowly than dedicated inference engines, so treat the speed as a reference only.

Answers to Our Japanese Questions (Verbatim)

We ask every model the same five questions in Japanese under the conditions above (temperature 0, thinking turned off where the chat template allows it, up to 1024 tokens) and publish the answers unedited. We do not grade them; where a question has a single correct answer, we show it for comparison.

Summarization

次の記事を3文で要約してください。

みどり市は今年の春、市内に三つある公立図書館のうち最も古い中央図書館の改修を終え、十月一日に再開館した。改修は約一年半にわたり、総事業費は約十二億円。このうち約四億円は国の補助金でまかなわれた。

最大の変化は、閲覧席の配置である。以前は館内の座席の大半が学習用の机で、試験期間になると中高生で埋まり、高齢の利用者や小さな子ども連れが座れないという苦情が毎年寄せられていた。改修後は座席数を百二十席から二百席に増やしたうえで、学習用の机を三階にまとめ、一階は会話ができる「にぎわいフロア」、二階は静かに本を読む「閲覧フロア」に分けた。一階には予約制のグループ学習室が四室と、飲み物を持ち込めるカフェスペースも設けられた。

蔵書の面でも見直しが進んだ。貸出の少ない古い資料約三万冊を市内の他の館や保存書庫に移し、代わりに子ども向けの絵本と、介護・子育て・仕事探しといった暮らしに関わる実用書を約一万五千冊増やした。電子書籍の貸出サービスも始まり、利用登録をすれば自宅から一度に三冊まで、二週間借りられる。

建物そのものにも手が入った。築四十年を超えていた空調設備を入れ替え、屋上には太陽光パネルを設置して、館内で使う電力の約二割をまかなえるようにした。車いすの利用者から要望の多かったエレベーターは一基から二基に増え、書架の間の通路も以前より三十センチ広げられた。入口には段差のないスロープが新たに設けられ、授乳室と多目的トイレも各階に一つずつ置かれた。

一方で課題も残る。駐車場は改修前と同じ四十台のままで、週末には近くの道路に順番待ちの車が並んだ。市は隣接する市有地を臨時駐車場として開放したが、恒久的な対策は決まっていない。また、開館時間を平日の夜九時まで延ばしたことで職員の負担が増えており、職員団体は人員の追加を求めている。

利用者の受け止めはおおむね好意的だ。週に二度、孫を連れて来るという七十代の女性は「前は子どもが声を出すと周りの目が気になったが、一階なら気兼ねなく絵本を読んであげられる」と話す。一方、受験を控えた高校生の中には「学習席が三階だけになり、夕方はすぐに満席になる」と不満を漏らす人もいる。図書館は、学習席の空き状況をホームページで確認できる仕組みを年内に導入する予定だ。

再開館に合わせて、催しも増えた。毎週土曜の午前には読み聞かせ会、月に一度は地元の農家や商店主を講師に招く「まちの先生」講座が開かれる。十一月には、市内の中学生が選んだ本を紹介する展示も予定されている。

館長の山本和子さんは「本を借りる場所から、人が集まって過ごす場所へ変えたかった。再開館からの一か月で来館者は以前の一・八倍になった」と話す。市は来年度、残る二つの図書館についても利用者への聞き取りを行い、改修の必要性を検討する方針だ。

(Summarize the following article in three sentences. (The article, about 1,200 Japanese characters, describes the renovation and reopening of a city’s central library: the budget, the new floor layout and seating, changes to the collection and e-book lending, building and accessibility work, remaining issues such as parking and staff workload, users’ reactions, new events, and the city’s next steps.))

みどり市の中央図書館は約1年半の改修を経て再開し、座席を120席から200席に増やし、階層ごとの利用目的を分離するなど、利用者のニーズに応じた改善が図られた。蔵書は貸出の少ない資料を削減し、実用書や絵本を増やし、電子書籍貸出や太陽光発電など環境面でも手が入ったが、駐車場不足や職員負担といった課題は残っている。再開後の一か月で来館者は前年比1.8倍に増加し、図書館は「人が集まって過ごす場所」へと変貌しつつ、残る二つの図書館の改修も検討される。

Formal business email

取引先の担当者に、来週火曜日の打ち合わせを水曜日に変更してほしいとお願いするメールを、丁寧な敬語で書いてください。

(Write an email in polite Japanese (keigo) asking a client’s contact person to move next Tuesday’s meeting to Wednesday.)

**件名：** 次回打ち合わせ日程の変更のお願い（来週火曜日の打ち合わせを水曜日に変更）

---

拝啓　平素より格別のご高配を賜り、誠にありがとうございます。

このたびは突然の変更で恐縮ではございますが、来週の打ち合わせが予定されております**火曜日（○月○日）**の件につきまして、**水曜日（○月○日）**に変更させていただきたく存じます。

つきましては、下記のとおりご都合のよいお時間にてご検討いただけますと幸いに存じます。

| 項目 | 内容 |
|------|------|
| 変更前 | 火曜日（○月○日） |
| 変更後 | 水曜日（○月○日） |
| 目的 | 打ち合わせ（議題は不变） |
| 場所・方法 | 従来通り（オンライン／会議室） |

ご多忙の折から恐れ入りますが、**水曜日の日程で問題がなければご確認のほど、よろしくお願い申し上げます**。  
もし水曜日でもご都合がよろしくない場合は、代替案をご提示いただけますと幸いです。

何卒ご理解のほど、よろしくお願い申し上げます。

敬具

---

**株式会社○○　○○ 部　○○ 様**  
担当：○○（○○）  
電話番号：○○-○○-○○  
メールアドレス：○○@○○.co.jp

Word problem

りんごが1個120円、みかんが1個45円です。りんごを3個とみかんを8個買って、1,000円札で払いました。おつりはいくらですか。考え方も示してください。

(Apples cost 120 yen each and mandarins cost 45 yen each. You buy 3 apples and 8 mandarins and pay with a 1,000-yen bill. How much change do you get? Show your reasoning.)

(The model spent the whole token budget thinking and gave no answer.)

Reference: The correct answer is 280 yen (120×3 = 360, 45×8 = 360, 1,000 − 720 = 280).

Knowledge

日本の都道府県のうち、海に面していない県をすべて挙げてください。

(List all of Japan’s prefectures that do not border the sea.)

(The model spent the whole token budget thinking and gave no answer.)

Reference: The correct answer is eight prefectures: Tochigi, Gunma, Saitama, Yamanashi, Nagano, Gifu, Shiga and Nara.

Constrained writing

「春」をテーマにした俳句を3つ、五・七・五の音数を守って作ってください。それぞれに一言の説明を付けてください。

(Write three haiku on the theme of spring, keeping the 5-7-5 syllable pattern, each with a one-line explanation.)

(The model spent the whole token budget thinking and gave no answer.)

Setup, Download and Run Steps (What We Actually Ran)

We split the work into two containers on Modal, both built from the Debian (slim) image with Python 3.12. The container that downloads the files never runs the publisher’s code; the container that runs it has no network access. These are the packages we installed (without version pins; the versions actually installed are listed below).

# 1. Environment for downloading (no GPU)
python -m pip install huggingface_hub

# 2. Environment for running (GPU, network blocked)
python -m pip install torch transformers accelerate safetensors sentencepiece tiktoken protobuf einops compressed-tensors

We then called download(job) in the first container and run(job) in the second one from the script below (remote_code_remote.py, written by us; full text). Files go under /models/<dir>. Comments are in Japanese as in our repository:

"""
Modal のコンテナの中で動く、配布元のコード(trust_remote_code)による実測(2026-09-30)。
lmw/lab/gpu_modal.py の download_weights / run_remote_code から呼ばれる。**標準ライブラリだけを先頭で読み**、
huggingface_hub・torch・transformers は各関数の中で読む(コンテナごとに入っているものが違う)。

【2つのコンテナに分ける】
- download: GPU の無いコンテナ・ネットワークあり。ファイルを落とすだけで、**配布元のコードは実行しない**
  (import もしない)。重みは safetensors だけ(呼び出し側が渡すファイル名の一覧に pickle は入らない)。
- run: GPU のコンテナ・**ネットワーク遮断**(block_network)・Modal の機能も使えない(restrict_modal_access)・
  Volume は読み取り専用。ここでだけ配布元のコードを実行する。外へ送ることも、ほかのコンテナを起動することもできない。

記事執筆用の LLM は呼ばない。回答の良し悪しも判定しない(原文を返すだけ)。
"""
import json
import os
import re
import platform
import time
from datetime import datetime, timezone

MOUNT = "/models"
# コンテナのイメージ(lmw/lab/gpu_modal.py が使い、記事の手順にも同じ値を載せる)
PYTHON_VERSION = "3.12"
DOWNLOAD_PACKAGES = ("huggingface_hub",)
# compressed-tensors は量子化済みの配布(NVFP4 等。2026-10-01、K2-Horizon-32B-NVFP4)を transformers が読むために入れる
RUN_PACKAGES = ("torch", "transformers", "accelerate", "safetensors", "sentencepiece", "tiktoken", "protobuf", "einops",
                "compressed-tensors")

def installed_packages() -> list[str]:
    """このコンテナに入っているパッケージの版(pip freeze と同じ形)。記事の手順に載せる。"""
    from importlib import metadata
    seen = {}
    for dist in metadata.distributions():
        name = dist.metadata["Name"]
        if name and name.lower() not in seen:
            seen[name.lower()] = f"{name}=={dist.version}"
    return sorted(seen.values(), key=str.lower)

def _log(t0: float, message: str) -> None:
    print(f"[rc {time.time() - t0:6.0f}s] {message}", flush=True)

def download(job: dict) -> dict:
    """job: {"repo", "revision", "files": [...], "dir"}。戻り値: {"bytes", "seconds", "packages"}。"""
    from huggingface_hub import hf_hub_download
    t0 = time.time()
    target = os.path.join(MOUNT, job["dir"])
    os.makedirs(target, exist_ok=True)
    total = 0
    for name in job["files"]:
        path = hf_hub_download(job["repo"], name, revision=job["revision"], local_dir=target)
        total += os.path.getsize(path)
    seconds = round(time.time() - t0)
    _log(t0, f"ダウンロード {total / 1024 ** 3:.1f}GB・{seconds}秒({total / 1024 ** 2 / max(seconds, 1):.0f}MB/秒)")
    return {"bytes": int(total), "seconds": int(seconds), "packages": installed_packages()}

def _chat_ids(tok, text: str):
    messages = [{"role": "user", "content": text}]
    try:
        out = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
                                      return_tensors="pt")
    except TypeError:
        out = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
    # 版によっては dict(BatchEncoding)が返る
    return out["input_ids"] if hasattr(out, "keys") else out

# 思考の目印。<think> に限らない(K2-Horizon は <ifm|think>・<ifm|think_fast> と </ifm|think>。2026-09-30)
_THINK_OPEN_RE = re.compile(r"<(?!/)([^<>\s]*think[^<>\s]*)>")

def split_thinking(generated: str, prompt_text: str = "") -> tuple[str, str]:
    """
    生成した文字列(特別な記号を残したもの)を (回答, 思考) に分ける。llama-server の reasoning_content と同じ扱い。
    チャットテンプレートがプロンプトの末尾で思考の目印を開けている場合は、生成は思考の途中から始まる。
    閉じる目印は、開けた目印の名前から決める(<X> → </X>。ただし think_fast 等の変種は </…think> でも閉じる)。
    2026-09-30、特別な記号ごと消して文字にし、次に <think> だけを前提にしたため、K2-Horizon の英語の思考が
    2度「回答」として記録された。
    """
    text = generated
    opened = None
    tail = prompt_text.rstrip()
    for m in _THINK_OPEN_RE.finditer(tail):
        if m.end() == len(tail):
            opened = m
    if opened:
        text = opened.group(0) + text
    m = _THINK_OPEN_RE.search(text)
    if not m:
        return text, ""
    before, rest = text[:m.start()], text[m.end():]
    name = m.group(1)
    closers = [f"</{name}>"]
    if "|" in name or "_" in name:
        # 変種(think_fast)の閉じ方が本体(</ifm|think>)と同じテンプレートがある
        closers.append("</" + re.sub(r"(think)[^|<>]*$", r"\1", name) + ">")
    found = [(rest.find(c), c) for c in closers if c in rest]
    if found:
        at, closer = min(found)
        return before + rest[at + len(closer):], rest[:at]
    return before, rest

def run(job: dict) -> dict:
    """
    job: {"dir", "prompts": [{"id", "ja"}], "max_tokens"}。
    戻り値は記録の一部。チャットテンプレートが無ければ {"no_chat_template": True}。
    """
    import torch
    import transformers
    from transformers import AutoModelForCausalLM, AutoTokenizer
    os.environ["HF_HUB_OFFLINE"] = "1"
    os.environ["TRANSFORMERS_OFFLINE"] = "1"
    t0 = time.time()
    path = os.path.join(MOUNT, job["dir"])
    gpu = torch.cuda.get_device_name(0)
    vram_total = torch.cuda.get_device_properties(0).total_memory
    _log(t0, f"GPU: {gpu}・transformers {transformers.__version__}・torch {torch.__version__}")
    # 版は str にする(torch.__version__ は torch の型で、torch の無いホストで結果を読めない。2026-09-30、1本ぶん失った)
    record = {"gpu": gpu, "vram_total": int(vram_total), "transformers": str(transformers.__version__),
              "torch": str(torch.__version__), "dtype": "bfloat16", "packages": installed_packages(),
              "python": platform.python_version(),
              "measured_at": datetime.now(timezone.utc).isoformat()}
    # 量子化済みの配布なら、その形式を記録する(記事に「BF16 のまま」と書かない)
    with open(os.path.join(path, "config.json"), encoding="utf-8") as f:
        quant = json.load(f).get("quantization_config") or {}
    if quant:
        groups = list((quant.get("config_groups") or {}).values())
        record["quantization"] = {"quant_method": quant.get("quant_method"),
                                  "format": quant.get("format") or (groups[0].get("format") if groups else None),
                                  "weights_bits": ((groups[0].get("weights") or {}).get("num_bits") if groups else None)}
        _log(t0, f"量子化済みの配布: {record['quantization']}")
    tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True, local_files_only=True)
    if not getattr(tok, "chat_template", None):
        _log(t0, "チャットテンプレートがありません")
        return plain({**record, "no_chat_template": True})
    model = AutoModelForCausalLM.from_pretrained(path, trust_remote_code=True, local_files_only=True,
                                                 use_safetensors=True, dtype=torch.bfloat16,
                                                 device_map="cuda")
    model.eval()
    torch.cuda.synchronize()
    # 読み込んだ後の重みの持ち方(型ごとの要素数と、重みを持つ末端の層のクラス)。量子化済みの配布が 4bit のまま
    # 読まれたのか、BF16 へ展開されたのかを、推測ではなく数えて記録する
    dtypes, classes = {}, {}
    for tensor in list(model.parameters()) + list(model.buffers()):
        name = str(tensor.dtype).replace("torch.", "")
        dtypes[name] = dtypes.get(name, 0) + int(tensor.numel())
    for module in model.modules():
        if not list(module.children()) and list(module.parameters(recurse=False)):
            cls = type(module).__name__
            classes[cls] = classes.get(cls, 0) + 1
    record["loaded_weights"] = {"dtypes": dtypes, "leaf_classes": classes}
    record["vram_loaded"] = int(torch.cuda.memory_allocated())
    record["load_sec"] = round(time.time() - t0)
    _log(t0, f"読み込み: VRAM {record['vram_loaded'] / 1024 ** 3:.1f}GB・{record['load_sec']}秒")
    torch.cuda.reset_peak_memory_stats()
    outputs = []
    # 打ち切り(Modal の timeout)の前に、答えた分だけで返す。打ち切られると答えた分も全て失う
    # 次の1問が最長(max_tokens)まで書いたときの時間を、ここまでで最も遅かった速さで見積もって判定する
    stop_at = t0 + job.get("answer_budget_sec", 10 ** 9)
    for prompt in job["prompts"]:
        slowest = min((o["gen_tps"] for o in outputs if o.get("gen_tps")), default=None)
        worst = job["max_tokens"] / slowest if slowest else 0
        if time.time() + worst > stop_at:
            _log(t0, f"時間の上限が近いため、残りの問い({prompt['id']} から)は尋ねません")
            record["unanswered"] = [p["id"] for p in job["prompts"][len(outputs):]]
            break
        ids = _chat_ids(tok, prompt["ja"]).to("cuda")
        started = time.time()
        with torch.no_grad():
            generated = model.generate(ids, max_new_tokens=job["max_tokens"], do_sample=False)
        torch.cuda.synchronize()
        seconds = time.time() - started
        new = generated[0][ids.shape[1]:]
        count = int(new.shape[0])
        # 特別な記号(<think> 等)を残して文字にし、思考と回答を分けてから、残りの特別な記号を消す
        prompt_text = tok.decode(ids[0][-8:], skip_special_tokens=False)
        answer, thinking = split_thinking(tok.decode(new, skip_special_tokens=False), prompt_text)
        for special in tok.all_special_tokens:
            answer = answer.replace(special, "")
        outputs.append({
            "id": prompt["id"],
            "content": answer.strip(),
            "reasoning_chars": len(thinking.strip()),
            "finish_reason": "length" if count >= job["max_tokens"] else "stop",
            "completion_tokens": count,
            "gen_tps": round(count / seconds, 2) if seconds > 0 else None,
            "prompt_tokens": int(ids.shape[1]),
        })
        _log(t0, f"回答 {prompt['id']}: {count}トークン・{outputs[-1]['gen_tps']} トークン/秒")
    record["vram_peak"] = int(torch.cuda.max_memory_allocated())
    record["outputs"] = outputs
    gen = sorted(o["gen_tps"] for o in outputs if o.get("gen_tps"))
    if gen:
        record["tg_tps"] = gen[len(gen) // 2]
    return plain(record)

def plain(value):
    """JSON に通して、標準の型だけにする(ホストには torch も transformers も無い)。"""
    return json.loads(json.dumps(value, default=str))

The end-of-turn marker of the chat template (<|ifm|im_end|>) that remained at the end of some answers was removed after recording; the script above does not remove it.

The job passed to download (as used):

{
  "repo": "IFM/K2-Horizon-32B-NVFP4",
  "revision": "71b07893498c52a6508bfce3e3ccb5456ba75328",
  "files": [
    "chat_template.jinja",
    "config.json",
    "configuration_k2_horizon.py",
    "generation_config.json",
    "model-00001-of-00002.safetensors",
    "model-00002-of-00002.safetensors",
    "model.safetensors.index.json",
    "modeling_k2_horizon.py",
    "tokenizer.json",
    "tokenizer_config.json"
  ],
  "dir": "run-1790872724"
}

The job passed to run (as used):

{
  "dir": "run-1790872724",
  "max_tokens": 1024,
  "answer_budget_sec": 3420,
  "prompts": [
    {
      "id": "summary",
      "ja": "次の記事を3文で要約してください。\n\nみどり市は今年の春、市内に三つある公立図書館のうち最も古い中央図書館の改修を終え、十月一日に再開館した。改修は約一年半にわたり、総事業費は約十二億円。このうち約四億円は国の補助金でまかなわれた。\n\n最大の変化は、閲覧席の配置である。以前は館内の座席の大半が学習用の机で、試験期間になると中高生で埋まり、高齢の利用者や小さな子ども連れが座れないという苦情が毎年寄せられていた。改修後は座席数を百二十席から二百席に増やしたうえで、学習用の机を三階にまとめ、一階は会話ができる「にぎわいフロア」、二階は静かに本を読む「閲覧フロア」に分けた。一階には予約制のグループ学習室が四室と、飲み物を持ち込めるカフェスペースも設けられた。\n\n蔵書の面でも見直しが進んだ。貸出の少ない古い資料約三万冊を市内の他の館や保存書庫に移し、代わりに子ども向けの絵本と、介護・子育て・仕事探しといった暮らしに関わる実用書を約一万五千冊増やした。電子書籍の貸出サービスも始まり、利用登録をすれば自宅から一度に三冊まで、二週間借りられる。\n\n建物そのものにも手が入った。築四十年を超えていた空調設備を入れ替え、屋上には太陽光パネルを設置して、館内で使う電力の約二割をまかなえるようにした。車いすの利用者から要望の多かったエレベーターは一基から二基に増え、書架の間の通路も以前より三十センチ広げられた。入口には段差のないスロープが新たに設けられ、授乳室と多目的トイレも各階に一つずつ置かれた。\n\n一方で課題も残る。駐車場は改修前と同じ四十台のままで、週末には近くの道路に順番待ちの車が並んだ。市は隣接する市有地を臨時駐車場として開放したが、恒久的な対策は決まっていない。また、開館時間を平日の夜九時まで延ばしたことで職員の負担が増えており、職員団体は人員の追加を求めている。\n\n利用者の受け止めはおおむね好意的だ。週に二度、孫を連れて来るという七十代の女性は「前は子どもが声を出すと周りの目が気になったが、一階なら気兼ねなく絵本を読んであげられる」と話す。一方、受験を控えた高校生の中には「学習席が三階だけになり、夕方はすぐに満席になる」と不満を漏らす人もいる。図書館は、学習席の空き状況をホームページで確認できる仕組みを年内に導入する予定だ。\n\n再開館に合わせて、催しも増えた。毎週土曜の午前には読み聞かせ会、月に一度は地元の農家や商店主を講師に招く「まちの先生」講座が開かれる。十一月には、市内の中学生が選んだ本を紹介する展示も予定されている。\n\n館長の山本和子さんは「本を借りる場所から、人が集まって過ごす場所へ変えたかった。再開館からの一か月で来館者は以前の一・八倍になった」と話す。市は来年度、残る二つの図書館についても利用者への聞き取りを行い、改修の必要性を検討する方針だ。"
    },
    {
      "id": "keigo",
      "ja": "取引先の担当者に、来週火曜日の打ち合わせを水曜日に変更してほしいとお願いするメールを、丁寧な敬語で書いてください。"
    },
    {
      "id": "arithmetic",
      "ja": "りんごが1個120円、みかんが1個45円です。りんごを3個とみかんを8個買って、1,000円札で払いました。おつりはいくらですか。考え方も示してください。"
    },
    {
      "id": "prefectures",
      "ja": "日本の都道府県のうち、海に面していない県をすべて挙げてください。"
    },
    {
      "id": "haiku",
      "ja": "「春」をテーマにした俳句を3つ、五・七・五の音数を守って作ってください。それぞれに一言の説明を付けてください。"
    }
  ]
}

accelerate==1.15.0 aiohappyeyeballs==2.6.1 aiohttp==3.12.7 aiosignal==1.3.2 annotated-doc==0.0.5 annotated-types==0.8.0 anyio==4.15.1 attrs==25.3.0 cbor2==5.7.0 certifi==2026.7.22 charset-normalizer==3.5.2 click==8.5.0 compressed-tensors==0.19.0 cuda-bindings==13.4.3 cuda-pathfinder==1.8.2 cuda-toolkit==13.0.3.0 einops==0.8.2 filelock==4.0.8 frozenlist==1.6.0 fsspec==2026.9.0 grpclib==0.4.8 h11==0.16.0 h2==4.2.0 hf-xet==1.6.0 hpack==4.1.0 httpcore==1.0.9 httpx==0.28.1 huggingface_hub==1.33.0 hyperframe==6.1.0 idna==3.20 Jinja2==3.1.6 loguru==0.7.3 markdown-it-py==4.2.0 MarkupSafe==3.0.3 mdurl==0.1.2 mpmath==1.3.0 multidict==6.4.4 networkx==3.7 numpy==2.5.3 nvidia-cublas==13.1.1.3 nvidia-cuda-cupti==13.0.85 nvidia-cuda-nvrtc==13.0.88 nvidia-cuda-runtime==13.0.96 nvidia-cudnn-cu13==9.24.0.43 nvidia-cufft==12.0.0.61 nvidia-cufile==1.15.1.6 nvidia-curand==10.4.0.35 nvidia-cusolver==12.0.4.66 nvidia-cusparse==12.6.3.3 nvidia-cusparselt-cu13==0.8.1 nvidia-nccl-cu13==2.30.7 nvidia-nvjitlink==13.4.92 nvidia-nvshmem-cu13==3.4.5 nvidia-nvtx==13.0.85 packaging==26.3 pip==25.1.1 propcache==0.3.1 protobuf==7.36.2 psutil==7.2.2 pydantic==2.13.5 pydantic_core==2.46.5 Pygments==2.21.0 PyYAML==6.0.3 regex==2026.9.29 requests==2.34.2 rich==15.0.0 safetensors==0.8.0 sentencepiece==0.2.2 setuptools==84.0.0 shellingham==1.5.4 sympy==1.14.0 tiktoken==0.14.0 tokenizers==0.23.2 torch==2.14.1 tqdm==4.70.1 transformers==5.18.0 triton==3.8.0 typer==0.27.2 typing-inspection==0.4.4 typing_extensions==4.16.0 urllib3==2.8.0 uv==0.7.19 wheel==0.45.1 yarl==1.20.0

aiohappyeyeballs==2.6.1 aiohttp==3.12.7 aiosignal==1.3.2 anyio==4.15.1 attrs==25.3.0 cbor2==5.7.0 certifi==2025.4.26 click==8.5.0 filelock==4.0.7 frozenlist==1.6.0 fsspec==2026.9.0 grpclib==0.4.8 h11==0.16.0 h2==4.2.0 hf-xet==1.6.0 hpack==4.1.0 httpcore2==2.13.1 httpx2==2.13.1 huggingface_hub==2.0.0 hyperframe==6.1.0 idna==3.20 multidict==6.4.4 packaging==26.3 pip==25.1.1 propcache==0.3.1 protobuf==6.31.1 PyYAML==6.0.3 tqdm==4.70.1 truststore==0.10.4 typing_extensions==4.16.0 uv==0.7.19 wheel==0.45.1 yarl==1.20.0

How to Get It

This model can be freely downloaded and used from the Hugging Face repository. It is published under the “Apache-2.0" license and is not a gated model requiring prior consent for use, meaning anyone can obtain it immediately.

The distribution format is safetensors, and it is compatible with vLLM, SGLang, and Hugging Face’s Transformers library as serving engines.

Below are usage methods and code examples for each engine provided by the release origin. Note that the following code examples are written based on the BF16 version (IFM/K2-Horizon-32B), but similar settings and parser designations are also recommended when running the NVFP4 version.

Serving with vLLM

Here is an example command for launching a server using vLLM. It is recommended to enable the k2_horizon reasoning parser for chat and the tool call parser for agents.

vllm serve IFM/K2-Horizon-32B \
  --revision main \
  --model-impl vllm \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon

Serving with SGLang

Here is an example command when using SGLang. This recipe is verified on a 2× H200 environment.

python3 -m sglang.launch_server \
  --model-path IFM/K2-Horizon-32B \
  --revision main \
  --tp 2 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --host 0.0.0.0 --port 30000

API Usage Example (Python)

Here is a Python code example to obtain answers with thought processes via the OpenAI-compatible API after starting the server. Recommended settings such as temperature=1.0, top_p=0.95, and reasoning_effort="high" are specified.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="IFM/K2-Horizon-32B",
    messages=[{"role": "user", "content": "Explain the result step by step."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Usage Example with Transformers

Here is a code example to load the model directly and generate text using Transformers (verified with version 5.15.0, PyTorch 2.13.0, and Safetensors 0.8.0).

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "IFM/K2-Horizon-32B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)

inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Related Articles

• K2-Horizon-375B-A23B-NVFP4 Text Generation Model: ~257GB Memory
• DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory
• Qwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory
• DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds

What to Read Next

• Find models by VRAM (This model needs at least 32GB) → VRAM quick reference
• Explore the same model family → K2-Horizon family overview (3 articles, 1 converted builds)
• Engines that run this model → vLLM
• What NVFP4 mean and where to get this model → NVFP4 / MXFP4 format guide and models / Quantization and model-format glossary
• Learn about the publisher → IFM (MBZUAI): models, licenses and articles

Sources

• IFM/K2-Horizon-32B-NVFP4 (Hugging Face)

Update History

• 2026-09-24: Rebuilt the article: Added a “can you run it locally?" verdict from the distribution formats, converted builds, engine registries, license terms and measured compression
• 2026-09-25: Changed the title to show what the article covers (VRAM requirements, file list, etc.).
• 2026-09-28: Added our own measurements: Japanese token efficiency.
• 2026-10-02: Added our own measurements: VRAM and answers to Japanese questions with the publisher’s own code.
• 2026-10-02: Changed the title to show what the article covers (VRAM requirements, file list, etc.).