This page collects what Local Model Watch measures and records itself, rather than what model cards say. We read the tokenizer and the GGUF header of each model we cover, run small models on our own CPU-only server, and keep daily records of Hugging Face and of each inference engine’s model registry. Models too large for that server are run on a rented cloud GPU, and we generate images and speech on the same server. Everything below is computed by code from those records; no language model writes or grades these figures. Each model’s article shows the same measurements in its \u201cOur Own Measurements\u201d section, including the verbatim answers to our five Japanese questions (temperature 0, up to 1024 tokens).<\/p>\n
From the models we actually ran on our own server (Neoverse-N1, 3 threads, no GPU, llama.cpp b11223, b11347, b11856, b12210), we pick one per purpose by speed, answers, memory and license. Only models whose license allows commercial use (with or without conditions) are included, and only those that answered the current set of our Japanese questions (13 models).<\/p>\n
Q2_K<\/code>, 12.9GB file \u2014 score 92, 13.4 Japanese chars\/s, peak memory 14.9GB, license apache-2.0 (Commercial use allowed). Highest score among models at usable speed. A 4-bit file would not fit our 13GB limit, so we measured Q2_K<\/code>; we have not measured it at 4 bits.<\/li>\n- Speed first<\/strong>: MiniCPM5-2B<\/a> \u2014
Q4_K_M<\/code>, 1.5GB file \u2014 score 58, 20.9 Japanese chars\/s, peak memory 2.9GB, license apache-2.0 (Commercial use allowed). Fastest among models scoring 50 or more.<\/li>\n- Peak memory up to 8GB<\/strong>: MiniCPM5-2B<\/a> \u2014
Q4_K_M<\/code>, 1.5GB file \u2014 score 58, 20.9 Japanese chars\/s, peak memory 2.9GB, license apache-2.0 (Commercial use allowed). Highest score at usable speed and 50+ points within 8GB.<\/li>\n- Peak memory up to 16GB<\/strong>: Nex-N2.5-mini<\/a> \u2014
Q2_K<\/code>, 12.9GB file \u2014 score 92, 13.4 Japanese chars\/s, peak memory 14.9GB, license apache-2.0 (Commercial use allowed). Highest score at usable speed and 50+ points within 16GB. A 4-bit file would not fit our 13GB limit, so we measured Q2_K<\/code>; we have not measured it at 4 bits.<\/li>\n<\/ul>\nWhat the numbers show<\/h3>\n\n- 12 of 13 models got the word problem right.<\/li>\n
- No model named the eight landlocked prefectures without omissions or mistakes; the closest was Nex-N2.5-mini (7 of 8, 1 wrong). This was the question that separated the models most.<\/li>\n
- 9 of 13 models kept the summary to the three sentences we asked for.<\/li>\n
- 5 of 12 models were below 5 chars\/s: vectionlabs_Salience-27B-R6-GGUF (1.6), Qwopus3.8-27B-Flash-GGUF (1.7), Qwen3.8-27B (1.2), Signal-3.8-27B-GGUF (1.2), olmo3-7b-sdf-sft-clean150 (4.4).<\/li>\n<\/ul>\n
All candidates<\/h3>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n