Local LLM News, Quick References and Our Own Measurements
Local Model Watch is a reference site for open-weight generative models (text, image, video and audio) you can run on your own hardware, in English and Japanese: articles on new releases, hub pages by VRAM, model family, publisher, format and inference engine, and measurements we take ourselves by running models on our own server. Memory requirements in every article are computed by this site from the actual distributed file sizes — see our editorial policy.
Start here
Local models by VRAM tier — 52 models listedSee our own measurements
Japanese token efficiency, GGUF internals, CPU runs and generated media for 47 modelsBrowse by model family
17 families with articles, hardware tables and tracked variantsSee what’s trending
Likes and downloads of 70 models we covered, updated weeklyBrowse by publisher
49 companies and labs: their Hugging Face models and our coverageBrowse by model format
8 formats such as GGUF, MLX and FP8: what they are and which models ship in themFollow inference engines and tools
Release history of 25 projects (llama.cpp, Ollama, vLLM, …)Browse by task
Image, audio, video and text models side by side — 65 modelsRead the glossaries
45 benchmarks explained, plus quantization and model formats
Everything that fits your GPU: 8GB / 12GB / 16GB / 24GB
Quantization and model-format glossary is also available.
Our latest measurements
Models we ran ourselves on our server’s CPU (Neoverse-N1, no GPU) with the official llama.cpp builds. These are CPU figures, not a guide to GPU speed. Each article shows the answers verbatim.
→ Scroll horizontally to see all columns
| Model | File (quant) | Generation | In Japanese | Peak memory | Conditions | Measured |
|---|---|---|---|---|---|---|
| clef-flash Vision-Language Model: Our Test Answers, 4GB+ VRAM | Cloudflare_clef-flash-Q4_K_M.gguf(Q4_K_M) |
3.9 tok/s | 7.1 chars/s | 11.8GB | llama.cpp b11347, 3 threads | 2026-10-03 |
| MiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM | MiMo-V2.6-Distill-Qwen-9B-Q8_0.gguf(Q8_0) |
3.7 tok/s | 6.8 chars/s | 14.8GB | llama.cpp b11223, 3 threads | 2026-09-30 |
| Nex-N2.5-mini Text Generation Model: Our Test Answers, 16GB+ VRAM | nex-agi_Nex-N2.5-mini-Q2_K.gguf(Q2_K) |
7.3 tok/s | 13.4 chars/s | 14.9GB | llama.cpp b11223, 3 threads | 2026-09-29 |
| Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF: Our Test Answers | Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-Q3_K_S.gguf(Q3_K_S) |
6.2 tok/s | 11.0 chars/s | 14.9GB | llama.cpp b11223, 3 threads | 2026-09-29 |
| vectionlabs_Salience-27B-R6-GGUF: Our Test Answers, 12GB+ VRAM | vectionlabs_Salience-27B-R6-Q3_K_M.gguf(Q3_K_M) |
0.9 tok/s | 1.6 chars/s | 14.7GB | llama.cpp b11223, 3 threads | 2026-09-29 |
Models our CPU cannot run were run on a rented cloud GPU. Speeds are for that one GPU and engine, not for llama.cpp or other GPUs.
→ Scroll horizontally to see all columns
| Model | Engine | GPU | VRAM after loading | Peak VRAM | Generation | Measured |
|---|---|---|---|---|---|---|
| ELYZA Releases ELYZA-Thinking-1.0 32B/33B Reasoning Models | transformers | NVIDIA A100-SXM4-80GB | 59.9GB | 60.1GB | 13.7 tok/s | 2026-10-11 |
| K2-Horizon-32B-NVFP4: Our Test Answers, 32GB+ VRAM | transformers | NVIDIA A100-SXM4-80GB | 21.7GB | 71.3GB | 2.2 tok/s | 2026-10-02 |
| OrcaSAQ-2-27B Text Generation Model: Our Test Answers, 16GB+ VRAM | ExLlamaV3 1.5.3 | NVIDIA L4 | 13.7GB | 14.0GB | 17.2 tok/s | 2026-10-01 |
| Xing4.0-29B-A4B Text Generation Model: Our Test Answers, 24GB+ VRAM | transformers | NVIDIA A100-SXM4-80GB | 55.0GB | 55.8GB | 2.3 tok/s | 2026-09-30 |
All measurements and how we measure →
Video, images and audio we generated
We generate these ourselves from fixed prompts and sentences (no people, logos or reference audio) and show them in each article without picking or editing (videos are re-encoded to fit our upload limit). Each article also has the exact setup and run steps.
→ Scroll horizontally to see all columns
| Model | Kind | Files | Where we ran it | Generated |
|---|---|---|---|---|
| LongLive-Plug-Wan2.2-TI2V-5B-cfg: Our Generated Video, File List | Video | 4 clips | Cloud GPU (NVIDIA L40S) | 2026-10-02 |
| LongLive-Plug-Wan2.1-T2V-14B-few-step: Our Generated Video, File List | Video | 4 clips | Cloud GPU (NVIDIA L40S) | 2026-10-02 |
| Supra2-IMG Image Generation Model: Our Generated Images, File List | Images | 3 images | Our server’s CPU | 2026-09-30 |
| Irodori-TTS-v4.1-Anime: Our Generated Audio, File List | Speech | 13 clips | Our server’s CPU | 2026-09-30 |
Latest by category
New Models
Image, Video and Audio
Engines and Tools
Community
Weekly Roundup
Most read (past 28 days)
-
1
-
2
-
3
-
4Two Quantized Versions of Audio-Visual Video Generation Model FastH
Image, Video and Audio2026-10-06 · 62 views -
5
Last article published 2026-10-06 (JST).