Open-Weight Model Weekly: Ternary 27B Models and Apple Silicon Updates
This Week in Numbers
Counted by Local Model Watch from the articles published this week.
| Item | Count |
|---|---|
| Articles published | 39 |
| New Models | 12 |
| Engines and Tools | 10 |
| Technical Reports | 7 |
| Community | 4 |
| Image, Video and Audio | 3 |
| Companies and Funding | 3 |
| New models covered | 15 |
| fit in 8GB of VRAM (est.) | 4 |
| fit in 12GB of VRAM (est.) | 10 |
| fit in 16GB of VRAM (est.) | 10 |
| fit in 24GB of VRAM (est.) | 12 |
| parameters up to 4B | 3 |
| parameters 15–40B | 5 |
| parameters over 40B | 1 |
| parameters 4–15B | 2 |
| Converted builds appended to earlier articles | 15 (GGUF 9, MLX 4, GPTQ 1, NVFP4 1) |
| Most active publishers | bartowski (3), unslothai/unsloth (3), prism-ml (3) |
Trending models we did not cover separately
Hugging Face repositories that trended this week but did not get their own article. Listed for reference; we have not reviewed their model cards.
| Model | Likes | Downloads | Why no article |
|---|---|---|---|
| XingChen-AGI/Xing4.0-29B-A4B | 872 | 12,617 | publisher not on our notable list |
| dealignai/Bonsai-2-27B-Ternary-CRACK-GGUF | 108 | 25,385 | publisher not on our notable list |
| empero-ai/Qwen3.8-35B-A3B-Distill-GGUF | 107 | 52,476 | publisher not on our notable list |
| Altworld/Hemmingway-1 | 97 | 0 | publisher not on our notable list |
Patch releases we did not cover separately
Releases of watched projects that were patch-level or had short notes. Each project’s page lists every version.
Highlights of the Week
-
Succession of Ternary Quantization Models in the 27B Class
Prism ML released a batch of ternary quantization models based on Qwen3.8-27B across multiple formats, significantly expanding the options for running medium-sized models locally in environments with limited VRAM capacity. -
Expansion of Dedicated Inference and Training Environments for Apple Silicon and ARM64
Local execution environments for edge and non-x86 environments have been further improved, such as the release of the Qwen3.8 acceleration engine “Splash Engine" by LM Studio, and Windows ARM64 binaries by Unsloth. -
Synchronized Updates of Major Local Inference Backends and Infrastructure Tools
Core engines such as ggml, llama.cpp, llamafile, SGLang, and LocalAI were updated one after another, adding precision control API features and optimizing loading methods.
Trends by Category
Text Generation
This week stood out for high-efficiency model deployments incorporating extreme quantization and sparse attention techniques, centered around the 15B–40B medium parameter band.
Of particular note is the concentrated release of models using ternary weights. Starting with Ternary-quantized 27B model “Bonsai 2 27B GGUF" released, reports include Qwen3.8-27B-based ternary weight model “Bonsai 2 27B" MLX 2bit version released and the development version Ternary weight model test version “prism-ml/Ternary-Bonsai-2-27B-gguf-dev" released, advancing the practical application of 27B models capable of running in VRAM environments of 12GB or less. As a theoretical background, “Breaking the 1.58-Bit Barrier for Ternary LLMs" paper released, which outlines methods challenging the 1.58-bit limit, also became a hot topic (unconfirmed information).
Additionally, quantized versions of multimodal models and those optimized for reasoning have expanded, with Orion-26B-A4B-v1.1 GGUF quantized version released, multimodal support, Salience-27B-R6 GGUF quantized version released, reasoning-optimized 27B multimodal model, and in the ultra-large band, Multimodal foundational model “Intern-S2-397B" GGUF quantized version released being provided.
For on-device or specific-purpose lightweight models, releases included harshatheg/Qwen-2.5-1B-RLCD released: parallel constrained decoding for Apple Silicon, ultra-small Cactus Compute releases 8–29MB automated model “Needle 3", and OCR-focused Tencent releases document analysis model “WeVisDoc-2B/4B".
Image, Video, and Audio
In the media generation domain, enhancements to real-time performance and conversational capabilities are progressing.
In the audio sector, sound quality was improved via Real-audio tokenizer v4 for music generation model YuE2-3B released, alongside the provision of real-time conversational models integrating voice and video, such as inclusionAI releases full-duplex voice and video dialogue model “Realtime-Venus".
In the image sector, Image generation and editing model “Qwen-Image-2.1" released, supporting transparency processing and advanced multi-image editing and its smaller model Image generation model “Qwen-Image-2.1" released, scaled down to 7B with transparency processing support were released, improving open-weight image editing environments.
Engines and Tools
It was a week where environments were improved across a wide range of layers, from inference infrastructure to UI tools.
In low-layer inference engines, ggml v0.24.0 released: addition of precision control API and backend enhancements, llama.cpp v0.4.1 released: loading method changes and new model support, and llamafile 0.10.6 released, llama.cpp updates and enhanced CUDA/HIP graph features were successively published. On the serving side, Large-scale serving framework “SGLang v0.5.20" released was also updated.
In terms of hardware optimization, Local inference engine for Apple Silicon “Splash Engine" released, accelerates Qwen3.8 drew attention. Furthermore, Unsloth, the standard fine-tuning tool, announced Unsloth releases Windows ARM64 binary version in addition to “unsloth" v0.1.810-beta released: multi-user support and Docker overhaul and Unsloth v0.1.811-beta released, Docker, AMD, and multi-user support, improving ARM support and ease of use in container environments.
In frontend and UI-related news, koboldcpp v1.121 released: supports Minimax H3 media reference and SDUI LoRA selector, ComfyUI v0.36.0 released: supports general-purpose loops, Yue2, and Marigold v2, and Local AI execution engine “LocalAI v4.10.0" released, dashboard overhauled were released.
Related Articles
- llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts
- ggml v0.26.0 Released with Sparse Flash Attention & New Backends
- llama.cpp v0.5.0 Released with Backend and Server Upgrades
- Run GGUF Models Directly in Transformers with llama.cpp Support
This Week’s Articles
Compiled by Local Model Watch from the article log. Articles about the same story are merged into one row.
New Models
→ Scroll horizontally to see all columns
| Date | Model | Params | Smallest VRAM tier | License | Converted builds | Article |
|---|---|---|---|---|---|---|
| 2026-09-19 | Cactus-Compute/needle3 | — | 4GB | apache-2.0 | — | needle3 Text Generation Model: 4GB+ VRAM |
| 2026-09-18 | prism-ml/Ternary-Bonsai-2-27B-gguf | 27.8B | 8GB | apache-2.0 | 2 | Ternary-Bonsai-2-27B-gguf: Our Test Answers, 8GB+ VRAM |
| 2026-09-18 | tencent/WeVisDoc-4B | 4.4B | 4GB | apache-2.0 | 2 | Tencent Releases WeVisDoc-2B and WeVisDoc-4B Document Parsing Models |
| 2026-09-17 | harshatheg/Qwen-2.5-1B-RLCD | 1.5B | 4GB | apache-2.0 | — | Qwen-2.5-1B-RLCD Text Generation Model: 4GB+ VRAM |
| 2026-09-16 | EleutherAI/olmo3-7b-sdf-sft-clean150 | 7.3B | 24GB | apache-2.0 | — | EleutherAI Releases OLMo-3-7B Models for Reward-Hacking Research |
| 2026-09-15 | bartowski/vectionlabs_Salience-27B-R6-GGUF | 27.8B | 12GB | apache-2.0 | — | vectionlabs_Salience-27B-R6-GGUF: Our Test Answers, 12GB+ VRAM |
| 2026-09-15 | bartowski/Intern-S2-397B-GGUF | 403.4B | — | apache-2.0 | — | Intern-S2-397B-GGUF Vision-Language Model: ~102GB Memory |
| 2026-09-15 | EleutherAI/bergson-wikitext-gpt2-leaderboard | — | — | apache-2.0 | — | EleutherAI Releases Bergson Leaderboard Baseline GPT-2 Model |
| 2026-09-14 | bartowski/TheDrummer_Orion-26B-A4B-v1.1-GGUF | 25.8B | 12GB | — | — | TheDrummer_Orion-26B-A4B-v1.1-GGUF Multimodal Model: 12GB+ VRAM |
| 2026-09-14 | tencent/Simple-Attention-Sparsification | 4.0B | 12GB | — | — | Simple-Attention-Sparsification Text Generation Model: 12GB+ VRAM |
Image, Video and Audio
→ Scroll horizontally to see all columns
| Date | Model | Params | Smallest VRAM tier | License | Converted builds | Article |
|---|---|---|---|---|---|---|
| 2026-09-20 | Qwen/Qwen-Image-2.1-PE-I2I | — | — | qwen-research | — | Qwen-Image-2.1 Released: Open-Weight Image Gen & Editing |
| 2026-09-19 | inclusionAI/Realtime-Venus | — | 24GB | apache-2.0 | — | Realtime-Venus Full-Duplex Audio-Video Dialog Model: 24GB+ VRAM |
| 2026-09-15 | Mothersuperior/yue2-mothersuperior-realaudio-tokenizer-v4 | 3.6B | 12GB | cc-by-nc-4.0 | — | yue2-mothersuperior-realaudio-tokenizer-v4: 12GB+ VRAM, File List |
Engines and Tools
Community
| Date | Article |
|---|---|
| 2026-09-21 | Hacker News Discussion on Open-Weight Qwen-Image-2.1 |
| 2026-09-17 | Breaking the 1.58-bit Barrier for Ternary LLMs with BITCOS |
| 2026-09-16 | Why Tech Community is Debating Bearish Views on LLMs |
| 2026-09-15 | Challenges of Migrating AI Agents to Local Ollama Environments |
Sources
- https://localmodelwatch.tsuchitsuchi.com/2026/09/14/tencent-sas-qwen3-checkpoints/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/14/orion-26b-a4b-v1-1-gguf/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/14/ggml-v0-24-0-released/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/15/llama-cpp-v0-4-1-released/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/15/migrating-prompts-from-opus-to-ollama/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/15/yue2-mothersuperior-realaudio-tokenizer-v4/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/15/eleutherai-bergson-wikitext-gpt2-leaderboard/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/15/intern-s2-397b-gguf/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/15/salience-27b-r6-gguf/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/16/koboldcpp-v1-121/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/16/llamafile-0106/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/16/comfyui-v0360/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/16/why-im-still-bearish-on-llms-after-navier-stokes/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/16/eleutherai-olmo-3-7b-sdf-sft-models/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/16/unsloth-windows-arm64/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/17/harshatheg-qwen-25-1b-rlcd/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/17/tensorrt-edge-llm-jetson-agx-thor-qwen3-6/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/17/breaking-the-158-bit-barrier-for-ternary-llms/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/17/glm-inference-infrastructure-and-argodrive/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/18/unsloth-v0-1-810-beta-update/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/18/tencent-wevisdoc-2b-4b/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/18/localai-v4100/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/18/lm-studio-introspection/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/18/bonsai-2-27b-gguf/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/19/unsloth-v0-1-811-beta/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/19/cactus-needle-3/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/19/sglang-v0520-released/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/19/realtime-venus/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/20/splash-engine-qwen3-apple-silicon/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/20/qwen-image-21/
- https://localmodelwatch.tsuchitsuchi.com/2026/09/21/qwen-image-21-2/