{"id":6735,"date":"2026-09-28T17:30:34","date_gmt":"2026-09-28T08:30:34","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/observations-en\/"},"modified":"2026-09-29T01:30:02","modified_gmt":"2026-09-28T16:30:02","slug":"observations-en","status":"publish","type":"page","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/observations-en\/","title":{"rendered":"Our Measurements: Japanese Token Efficiency, CPU Runs, GGUF Internals and Engine Support"},"content":{"rendered":"

This page collects what Local Model Watch measures and records itself, rather than what model cards say. We read the tokenizer and the GGUF header of each model we cover, run small models on our own CPU-only server, and keep daily records of Hugging Face and of each inference engine’s model registry. Everything below is computed by code from those records; no language model writes or grades these figures. Each model’s article shows the same measurements in its \u201cOur Own Measurements\u201d section, including the verbatim answers to our five Japanese questions (temperature 0, up to 1024 tokens).<\/p>\n

Japanese Token Efficiency<\/h2>\n

We count how many tokens each model’s tokenizer needs for a fixed text we wrote ourselves (876 Japanese characters across six genres) and for its English translation. Fewer tokens mean more Japanese fits in the same context length, and faster generation per character. Models that share a tokenizer are listed on one row; \u201creference\u201d marks well-known tokenizers we measure for comparison.<\/p>\n

\n

\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
#<\/th>\nTokens per 1,000 Japanese chars<\/th>\nRatio to English<\/th>\nVocabulary<\/th>\nModels using this tokenizer<\/th>\n<\/tr>\n<\/thead>\n
1<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n99,574<\/td>\nLLM-jp-3 (reference)<\/td>\n<\/tr>\n
2<\/td>\n546<\/td>\n0.99\u00d7<\/td>\n248,070\u301c248,077<\/td>\nQwopus3.8-27B-Flash-GGUF, Nex-N2.5-mini<\/a>, Nex-N2.5-Pro<\/a>, Edge0-35B-A3B-preview<\/a>, Signal-3.8-27B-GGUF<\/a>, Intern-S2-397B-GGUF<\/a>, vectionlabs_Salience-27B-R6-GGUF<\/a>, Ternary-Bonsai-2-27B-gguf<\/a>, MiMo-V2.6-Distill-Qwen-9B-GGUF<\/a>, Tev1-4B-experimental<\/a>, Tev1-0.8B-experimental<\/a>, Qwen3.8-27B<\/a>, LensVLM-9B<\/a>, Hemmingway-1<\/a>, OrcaSAQ-2-27B<\/a><\/td>\n<\/tr>\n
3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n262,144\u301c262,145<\/td>\nGemma 3 (reference), Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/a>, TheDrummer_Orion-26B-A4B-v1.1-GGUF<\/a><\/td>\n<\/tr>\n
4<\/td>\n565<\/td>\n1.04\u00d7<\/td>\n250,624<\/td>\nK2-Horizon-MoVA-36B-A4B-GGUF, K2-Horizon-375B-A23B-NVFP4<\/a>, K2-Horizon-32B-NVFP4<\/a><\/td>\n<\/tr>\n
5<\/td>\n642<\/td>\n1.15\u00d7<\/td>\n125,017<\/td>\nLFM2.5-VL-3B-DSpark<\/a><\/td>\n<\/tr>\n
6<\/td>\n666<\/td>\n1.21\u00d7<\/td>\n130,560<\/td>\nMiniCPM5-2B<\/a><\/td>\n<\/tr>\n
7<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n151,665\u301c151,675<\/td>\nQwen3 (reference), Simple-Attention-Sparsification<\/a>, Qwen-2.5-1B-RLCD<\/a>, MiMo-V2.6-Flash-RL-GGUF<\/a>, MiMo-V2.6-Pro-RL<\/a>, MiMo-V2.6-Flash-MOPD<\/a><\/td>\n<\/tr>\n
8<\/td>\n705<\/td>\n1.29\u00d7<\/td>\n129,280<\/td>\nDeepSeek-V4.1-Flash<\/a>, DeepSeek-V4-Pro-0813-nvfp4-DSpark<\/a>, DeepSeek-V4-Pro-0813<\/a><\/td>\n<\/tr>\n
9<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n128,256<\/td>\nLlama 3.2 (reference)<\/td>\n<\/tr>\n
10<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n200,019<\/td>\ngpt-oss (reference)<\/td>\n<\/tr>\n
11<\/td>\n1,119<\/td>\n2.05\u00d7<\/td>\n100,278<\/td>\nolmo3-7b-sdf-sft-clean150<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Running Models on a CPU Only<\/h2>\n

Measured on our own server: Neoverse-N1, 3 threads, no GPU, with the official llama.cpp builds (b11223). Speeds come from llama-bench (512-token prompt, 128-token generation); peak memory is measured while answering our Japanese questions with a 4,096-token context and includes the memory-mapped model file. Each model’s article shows its answers verbatim.<\/p>\n

\n

\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\n\n\n\n\n\n
Model<\/th>\nQuant<\/th>\nFile<\/th>\nPrompt<\/th>\nGeneration<\/th>\nIn Japanese<\/th>\nPeak memory<\/th>\nMeasured<\/th>\n<\/tr>\n<\/thead>\n
MiniCPM5-2B<\/a><\/td>\nQ4_K_M<\/code><\/td>\n1.45GB<\/td>\n34.4 tok\/s<\/td>\n13.9 tok\/s<\/td>\n20.9 chars\/s<\/td>\n2.9GB<\/td>\n2026-09-28<\/td>\n<\/tr>\n
Qwen3-4B (reference)<\/td>\nQ4_K_M<\/code><\/td>\n2.33GB<\/td>\n18.9 tok\/s<\/td>\n6.8 tok\/s<\/td>\n\u2014<\/td>\n5.7GB<\/td>\n2026-09-28<\/td>\n<\/tr>\n
olmo3-7b-sdf-sft-clean150<\/a><\/td>\nQ4_K_M<\/code><\/td>\n4.16GB<\/td>\n11.3 tok\/s<\/td>\n4.9 tok\/s<\/td>\n4.4 chars\/s<\/td>\n12.4GB<\/td>\n2026-09-28<\/td>\n<\/tr>\n
Qwen3-8B (reference)<\/td>\nQ4_K_M<\/code><\/td>\n4.68GB<\/td>\n10.4 tok\/s<\/td>\n4.6 tok\/s<\/td>\n\u2014<\/td>\n9.5GB<\/td>\n2026-09-28<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Quality Loss by Quantization<\/h2>\n

For each model we download its quantizations one at a time and compare them with the F16 (or Q8_0) file on a fixed Japanese text \u2014 the opening of Natsume Soseki’s Botchan<\/em> (public domain, from Aozora Bunko) \u2014 using llama.cpp’s KL-divergence mode (context 512 tokens). Each cell is how often the most likely next token matches the baseline. Botchan<\/em> is famous and may be in the training data, so compare quantizations of the same model rather than models with each other.<\/p>\n

\n

\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\n\n\n\n
Model<\/th>\nBaseline<\/th>\nQ2_K<\/th>\nQ3_K_M<\/th>\nIQ4_XS<\/th>\nQ4_K_M<\/th>\nQ5_K_M<\/th>\nQ6_K<\/th>\n<\/tr>\n<\/thead>\n
Qwen3-4B (reference)<\/td>\nQ8_0<\/code><\/td>\n54.1%<\/td>\n72.7%<\/td>\n83.0%<\/td>\n83.9%<\/td>\n90.3%<\/td>\n91.7%<\/td>\n<\/tr>\n
MiniCPM5-2B<\/a><\/td>\nQ8_0<\/code><\/td>\n37.4%<\/td>\n63.6%<\/td>\n76.9%<\/td>\n81.5%<\/td>\n89.1%<\/td>\n90.8%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Inside the GGUF Files<\/h2>\n

We read only the header of each GGUF file (not the weights) with HTTP range requests. Of 18 files, 12 record that they were quantized with an importance matrix (imatrix), and 18 ship a chat template that mentions tool calls. The average bits per weight is the file’s data size divided by the number of weights.<\/p>\n

\n

\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Model<\/th>\nFile<\/th>\nAvg bits<\/th>\nMain types<\/th>\nimatrix<\/th>\nContext<\/th>\nTools in template<\/th>\n<\/tr>\n<\/thead>\n
Hemmingway-1<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.10<\/td>\nQ4_K 75% \/ Q6_K 18%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
LensVLM-9B<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.21<\/td>\nQ4_K 72% \/ Q6_K 21%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
Xing4.0-29B-A4B<\/a><\/td>\nIQ4_NL<\/code> (XingChen-AGI)<\/td>\n5.15<\/td>\nIQ4_NL 93% \/ BF16 5%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
DeepSeek-V4-Pro-0813<\/a><\/td>\nQ4_K_M<\/code> (DevQuasar)<\/td>\n4.84<\/td>\nQ4_K 84% \/ Q6_K 16%<\/td>\nno<\/td>\n1,048,576<\/td>\nyes<\/td>\n<\/tr>\n
Qwen3.8-27B<\/a><\/td>\nQ4_K_M<\/code> (unsloth)<\/td>\n4.82<\/td>\nIQ4_XS 33% \/ Q4_K 27%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
plamo-3-610m-fin-instruct<\/a><\/td>\nQ4_K_M<\/code> (mradermacher)<\/td>\n5.13<\/td>\nQ4_K 70% \/ Q6_K 30%<\/td>\nno<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
Tev1-4B-experimental<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.15<\/td>\nQ4_K 65% \/ Q5_K 15%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
olmo3-7b-sdf-sft-clean150<\/a><\/td>\nQ4_K_M<\/code> (EleutherAI)<\/td>\n4.90<\/td>\nQ4_K 81% \/ Q6_K 19%<\/td>\nno<\/td>\n65,536<\/td>\nyes<\/td>\n<\/tr>\n
vectionlabs_Salience-27B-R6-GGUF<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.10<\/td>\nQ4_K 75% \/ Q6_K 18%<\/td>\nyes<\/td>\n1,048,576<\/td>\nyes<\/td>\n<\/tr>\n
Intern-S2-397B-GGUF<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.05<\/td>\nQ4_K 77% \/ Q6_K 17%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
TheDrummer_Orion-26B-A4B-v1.1-GGUF<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.69<\/td>\nQ4_K 52% \/ Q8_0 20%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
Signal-3.8-27B-GGUF<\/a><\/td>\nQ4_K_M<\/code> (agentionai)<\/td>\n4.97<\/td>\nQ4_K 79% \/ Q6_K 20%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.77<\/td>\nQ4_K 51% \/ Q8_0 22%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
Nex-N2.5-Pro<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.06<\/td>\nQ4_K 78% \/ Q6_K 17%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
Nex-N2.5-mini<\/a><\/td>\nQ4_K_M<\/code> (bartowski)<\/td>\n5.15<\/td>\nQ4_K 76% \/ Q6_K 12%<\/td>\nyes<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n
MiniCPM5-2B<\/a><\/td>\nQ4_K_M<\/code> (openbmb)<\/td>\n4.95<\/td>\nQ4_K 78% \/ Q6_K 22%<\/td>\nno<\/td>\n131,072<\/td>\nyes<\/td>\n<\/tr>\n
K2-Horizon-MoVA-36B-A4B-GGUF<\/td>\nQ4_K_M<\/code> (IFM)<\/td>\n4.78<\/td>\nQ4_K 87% \/ Q6_K 13%<\/td>\nno<\/td>\n524,288<\/td>\nyes<\/td>\n<\/tr>\n
Qwopus3.8-27B-Flash-GGUF<\/td>\nQ4_K_M<\/code> (Jackrong)<\/td>\n4.92<\/td>\nQ4_K 80% \/ Q6_K 20%<\/td>\nno<\/td>\n262,144<\/td>\nyes<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

How Soon GGUF Versions Appear<\/h2>\n

The time from the creation of the original model’s Hugging Face repository to the creation of its first GGUF repository, for the models we covered (repository creation time, not public release: repositories are often created privately before release, and a negative value means the GGUF repository was created first, e.g. with early access). Median over 11 models: 15.2 hours<\/strong>.<\/p>\n

\n\n\n\n\n\n\n\n\n\n
Who made the first GGUF<\/th>\nModels<\/th>\nMedian hours<\/th>\n<\/tr>\n<\/thead>\n
bartowski<\/td>\n4<\/td>\n32.0<\/td>\n<\/tr>\n
mradermacher<\/td>\n2<\/td>\n23.5<\/td>\n<\/tr>\n
The original publisher<\/td>\n2<\/td>\n-13.4<\/td>\n<\/tr>\n
unsloth<\/td>\n2<\/td>\n101.3<\/td>\n<\/tr>\n
LiquidAI<\/td>\n1<\/td>\n0.0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Model<\/th>\nFirst GGUF<\/th>\nHours<\/th>\n<\/tr>\n<\/thead>\n
Hemmingway-1<\/a><\/td>\nbartowski\/Altworld_Hemmingway-1-GGUF<\/a><\/td>\n3.4<\/td>\n<\/tr>\n
LensVLM-9B<\/a><\/td>\nbartowski\/LensVLM-9B-GGUF<\/a><\/td>\n44.4<\/td>\n<\/tr>\n
Xing4.0-29B-A4B<\/a><\/td>\nXingChen-AGI\/Xing4.0-29B-A4B-GGUF<\/a><\/td>\n6.7<\/td>\n<\/tr>\n
DeepSeek-V4-Pro-0813<\/a><\/td>\nunsloth\/DeepSeek-V4-Pro-0813-GGUF<\/a><\/td>\n10.6<\/td>\n<\/tr>\n
Qwen3.8-27B<\/a><\/td>\nunsloth\/Qwen3.8-27B-GGUF<\/a><\/td>\n192.1<\/td>\n<\/tr>\n
LFM2.5-VL-3B-DSpark<\/a><\/td>\nLiquidAI\/LFM2.5-VL-3B-DSpark-GGUF<\/a><\/td>\n0.0<\/td>\n<\/tr>\n
plamo-3-610m-fin-instruct<\/a><\/td>\nmradermacher\/plamo-3-610m-fin-instruct-GGUF<\/a><\/td>\n15.2<\/td>\n<\/tr>\n
Tev1-4B-experimental<\/a><\/td>\nbartowski\/togethercomputer_Tev1-4B-experimental-GGUF<\/a><\/td>\n19.6<\/td>\n<\/tr>\n
Nex-N2.5-Pro<\/a><\/td>\nbartowski\/Nex-N2.5-Pro-GGUF<\/a><\/td>\n101.1<\/td>\n<\/tr>\n
Nex-N2.5-mini<\/a><\/td>\nmradermacher\/Nex-N2.5-mini-GGUF<\/a><\/td>\n31.8<\/td>\n<\/tr>\n
MiniCPM5-2B<\/a><\/td>\nopenbmb\/MiniCPM5-2B-GGUF<\/a><\/td>\n-33.5<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

How Soon Inference Engines Register New Architectures<\/h2>\n

Every day we fetch each engine’s model registry from its source code and note the first day each architecture name appears (tracking since 2026-09-18). For the models we covered, we compare that day with the day the model’s repository was created. \u201cNot registered\u201d means the name is not in the registry today, not that the model cannot run.<\/p>\n

\n

\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\n\n\n\n\n
Engine<\/th>\nRegistered when the model appeared<\/th>\nRegistered later<\/th>\nNot registered<\/th>\nAlready registered before tracking began<\/th>\n<\/tr>\n<\/thead>\n
llama.cpp<\/td>\n9<\/td>\n0<\/td>\n4<\/td>\n8<\/td>\n<\/tr>\n
vLLM<\/td>\n10<\/td>\n0<\/td>\n2<\/td>\n9<\/td>\n<\/tr>\n
MLX (mlx-lm)<\/td>\n6<\/td>\n0<\/td>\n9<\/td>\n6<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

How Often Inference Engines Release<\/h2>\n

Releases recorded from each engine’s GitHub repository in the 30 days up to 2026-09-28.<\/p>\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Project<\/th>\nReleases<\/th>\nMedian days between releases<\/th>\nLatest<\/th>\n<\/tr>\n<\/thead>\n
Unsloth<\/a><\/td>\n13<\/td>\n1.1<\/td>\n2026-09-28<\/td>\n<\/tr>\n
ExLlamaV3<\/a><\/td>\n9<\/td>\n3.3<\/td>\n2026-09-28<\/td>\n<\/tr>\n
ggml<\/a><\/td>\n6<\/td>\n0.8<\/td>\n2026-09-25<\/td>\n<\/tr>\n
Ollama<\/a><\/td>\n6<\/td>\n4.0<\/td>\n2026-09-23<\/td>\n<\/tr>\n
ComfyUI<\/a><\/td>\n3<\/td>\n5.7<\/td>\n2026-09-21<\/td>\n<\/tr>\n
KoboldCpp<\/a><\/td>\n3<\/td>\n5.4<\/td>\n2026-09-26<\/td>\n<\/tr>\n
llama.cpp<\/a><\/td>\n3<\/td>\n9.5<\/td>\n2026-09-24<\/td>\n<\/tr>\n
Open WebUI<\/a><\/td>\n3<\/td>\n10.8<\/td>\n2026-09-22<\/td>\n<\/tr>\n
InvokeAI<\/a><\/td>\n2<\/td>\n21.1<\/td>\n2026-09-28<\/td>\n<\/tr>\n
SGLang<\/a><\/td>\n2<\/td>\n13.8<\/td>\n2026-09-19<\/td>\n<\/tr>\n
vLLM<\/a><\/td>\n2<\/td>\n12.9<\/td>\n2026-09-22<\/td>\n<\/tr>\n
RamaLama<\/a><\/td>\n1<\/td>\n\u2014<\/td>\n2026-09-25<\/td>\n<\/tr>\n
whisper.cpp<\/a><\/td>\n1<\/td>\n\u2014<\/td>\n2026-09-11<\/td>\n<\/tr>\n
llamafile<\/a><\/td>\n1<\/td>\n\u2014<\/td>\n2026-09-16<\/td>\n<\/tr>\n
LocalAI<\/a><\/td>\n1<\/td>\n\u2014<\/td>\n2026-09-18<\/td>\n<\/tr>\n
TTS WebUI<\/a><\/td>\n1<\/td>\n\u2014<\/td>\n2026-09-01<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

This page collects what Local Model Watch measures and records itself, rather than what model cards say. We read the tokenizer and the GGUF header of each model we cover, run small models on our own CPU-only server, and keep daily records of Hugging Face and of each inference engine’s model registry. Everything below is […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-6735","page","type-page","status-publish","hentry"],"lang":"en","translations":{"en":6735,"ja":6734},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages\/6735","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=6735"}],"version-history":[{"count":3,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages\/6735\/revisions"}],"predecessor-version":[{"id":6867,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages\/6735\/revisions\/6867"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=6735"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}