llama.cpp v0.8.0: Multimodal Enhancements and LFM3 Support

At a Glance
| Item | Value |
|---|---|
| Repository | ggml-org/llama.cpp |
| Version | v0.8.0 |
| Published | 2026-10-28 |
| License | MIT |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
llama.cpp is an LLM (Large Language Model) inference engine written in C/C++. In this v0.8.0 update, multimodal capabilities have been significantly enhanced, including support for LiquidAI’s LFM3 family, new APIs for audio streaming, and the addition of new server endpoints that implement OpenAI’s real-time protocol.
The most significant change in this release is the introduction of a new mechanism for audio input and output. This enables more advanced multimodal use cases, such as real-time voice interaction and transcription.
Breaking Changes & Deprecations
This update includes changes to API design and CLI specifications. Developers and users utilizing these tools should be aware of the following changes:
| Item | Old Specification | New Specification |
|---|---|---|
| CLI Arguments | (May not be required) | --model or -hf is mandatory |
mtmd Audio Generation API |
Uses state_data buffer |
Uses mtmd_gen_state object |
no_alloc Setting |
Parameter within llama_hparams |
Explicit argument during load/context creation |
Changes to the CLI will affect users who launch models directly from the command line. Additionally, changes to the mtmd API will affect developers integrating audio generation features into their own applications. Regarding memory management changes, caution is required when handling low-level code that controls memory allocation.
Key Changes
Support for LiquidAI LFM3 Family and Many New Architectures
LiquidAI’s LFM3 series (text, vision, audio, omni) is now supported. In particular, LFM3-Audio-2B and LFM3-Omni-2B can process diverse inputs such as text, images, audio, and video, and feature ASR (Automatic Speech Recognition), TTS (Text-to-Speech), and voice chat capabilities. Furthermore, many new model architectures have been added, including DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, and Granite Switch Decision.
Introduction of Audio Streaming and Real-time APIs
As an expansion of multimodal functionality, an experimental mtmd_audio_stream API has been added to mtmd. This converts arriving PCM data into embedding rows available to the decoder. For server functionality, a /v1/realtime endpoint implementing OpenAI’s real-time protocol and a POST /tts endpoint for text-to-speech have been introduced. This enables real-time interaction and transcription using audio streaming I/O.
Memory Management and MoE Cache Optimization
The memory management mechanism has been revamped. no_alloc (the setting to perform no memory allocation) has been moved from model hyperparameters to an option during loading or context creation. This ensures that the no_alloc setting is properly respected in each memory cache, such as llama_kv_cache and llama_moe_cache. Additionally, for MoE (Mixture of Experts), the caching method for experts held in host memory has been changed from the traditional LRU to 50/50 SLRU. Furthermore, Sparse Attention (QSA/DSA) has been redesigned.
UI/UX Enhancements and Improved Model Management
The user interface has been significantly improved. A feature to search for and download models directly from the Hugging Face Hub has been added, making model management easier. Additionally, a dialog for searching past conversations and a “conversation compaction" feature to keep long conversations within context limits have been implemented. This makes local model usage and chat history management smoother.
Supported Models & Hardware
A very large number of new model architectures were supported in this update. The main supported models are as follows:
- LiquidAI LFM3 Family: LFM3-1.7B (text), LFM3-VL-2B (vision), LFM3-Audio-2B, LFM3-Omni-2B. Specifically, the Audio and Omni models support audio streaming I/O, ASR, TTS, and voice chat.
- LiquidAI Decision Models: d1-2B, d1-omni-2B.
- Other New Architectures: DeepSeek V4.1 Flash, SenseNova U1, Beam (Reflection), Kolibri-1 (Aleph-Alpha), Flint, Muse Spark Open (text, vision and audio), Granite Switch Decision (256-way decision head), H2O-Lightning-4B, MiniCPM-OCR 1.0/Jev, Soprano TTS.
- Embedding Models: Nemotron-3.5-Embed (supports non-standard GQA and BPE tokenizer).
Additionally, due to backend performance improvements, inference efficiency has been improved in specific hardware environments through the introduction of chunked graph launching in CUDA, graph replay in SYCL, Q2_K/Q3_K in OpenCL, Hexagon HTP reconstruction, and KleidiAI SVE kernels.
How to Get It
To install llama.cpp, please use the following commands depending on your environment:
# Example for updating the repository to the latest state and building
git pull && make
For detailed installation instructions, please check the official repository.
Speed on Our CPU
Measured by Local Model Watch on our CPU-only server (Neoverse-N1, 3 threads) with llama-bench (512-token prompt, 128-token generation), always on the same model: Qwen3-0.6B-Q4_K_M.gguf from unsloth/Qwen3-0.6B-GGUF (revision 50968a4468). Versions like v0.5.0 ship no binaries, so we used the first official ubuntu-arm64 build that contains the version; “+N” is how many commits that build is ahead of it. These are CPU figures, not a guide to GPU speed, and small differences can be run-to-run noise (± is the standard deviation).
| Version | Build measured | Prompt | Generation |
|---|---|---|---|
| v0.8.0 | b11960 | 127.9 ± 0.1 tok/s | 42.4 ± 0.4 tok/s |
| v0.7.0 | b11669 | 128.4 ± 0.1 tok/s | 43.1 ± 0.2 tok/s |
Related Articles
- llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts
- llama.cpp v0.5.0 Released with Backend and Server Upgrades
- llama.cpp v1.0.0 Released: Major Updates and New Features
- llama.cpp v0.9.0 Released: FP8 Quantization and More
What to Read Next
- Follow this tool → llama.cpp overview and release history (9 releases tracked)
- Other inference engines and runtimes → Ollama / vLLM / SGLang
- Engines mentioned in this article → ggml
Sources
- ggml-org/llama.cpp v0.8.0 (GitHub Release)
- PR #31252: LFM3 family support
- PR #31257: LFM3-Audio-2B and LFM3-Omni-2B support
- PR #31087: respect no_alloc in all memory caches
- PR #31141: move no_alloc out of llama_hparams
- PR #31200: remove obsolete n_pad from memory instances
- PR #31035: fix exhausting CPU and MEM caused by dry sequence breaker
- PR #30629: mtmd: add mtmd_audio_stream + support voxtral-realtime
- PR #30902: mtmd: refactor mtmd gen API, add state object
- PR #28607: mtmd : add Soprano audio generation

