Our Measurements: Japanese Token Efficiency, CPU Runs, GGUF Internals and Engine Support

September 28, 2026September 29, 2026

This page collects what Local Model Watch measures and records itself, rather than what model cards say. We read the tokenizer and the GGUF header of each model we cover, run small models on our own CPU-only server, and keep daily records of Hugging Face and of each inference engine’s model registry. Everything below is computed by code from those records; no language model writes or grades these figures. Each model’s article shows the same measurements in its “Our Own Measurements” section, including the verbatim answers to our five Japanese questions (temperature 0, up to 1024 tokens).

Japanese Token Efficiency

We count how many tokens each model’s tokenizer needs for a fixed text we wrote ourselves (876 Japanese characters across six genres) and for its English translation. Fewer tokens mean more Japanese fits in the same context length, and faster generation per character. Models that share a tokenizer are listed on one row; “reference” marks well-known tokenizers we measure for comparison.

Running Models on a CPU Only

Measured on our own server: Neoverse-N1, 3 threads, no GPU, with the official llama.cpp builds (b11223). Speeds come from llama-bench (512-token prompt, 128-token generation); peak memory is measured while answering our Japanese questions with a 4,096-token context and includes the memory-mapped model file. Each model’s article shows its answers verbatim.

Quality Loss by Quantization

For each model we download its quantizations one at a time and compare them with the F16 (or Q8_0) file on a fixed Japanese text — the opening of Natsume Soseki’s Botchan (public domain, from Aozora Bunko) — using llama.cpp’s KL-divergence mode (context 512 tokens). Each cell is how often the most likely next token matches the baseline. Botchan is famous and may be in the training data, so compare quantizations of the same model rather than models with each other.

Inside the GGUF Files

We read only the header of each GGUF file (not the weights) with HTTP range requests. Of 18 files, 12 record that they were quantized with an importance matrix (imatrix), and 18 ship a chat template that mentions tool calls. The average bits per weight is the file’s data size divided by the number of weights.

How Soon GGUF Versions Appear

The time from the creation of the original model’s Hugging Face repository to the creation of its first GGUF repository, for the models we covered (repository creation time, not public release: repositories are often created privately before release, and a negative value means the GGUF repository was created first, e.g. with early access). Median over 11 models: 15.2 hours.

How Soon Inference Engines Register New Architectures

Every day we fetch each engine’s model registry from its source code and note the first day each architecture name appears (tracking since 2026-09-18). For the models we covered, we compare that day with the day the model’s repository was created. “Not registered” means the name is not in the registry today, not that the model cannot run.

How Often Inference Engines Release

Releases recorded from each engine’s GitHub repository in the 30 days up to 2026-09-28.