llama.cpp v0.8.0: Multimodal Enhancements and LFM3 Support

llama.cpp v0.8.0: Multimodal Enhancements and LFM3 Support

At a Glance

Item Value
Repository ggml-org/llama.cpp
Version v0.8.0
Published 2026-10-28
License MIT
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

llama.cpp is an LLM (Large Language Model) inference engine written in C/C++. In this v0.8.0 update, multimodal capabilities have been significantly enhanced, including support for LiquidAI’s LFM3 family, new APIs for audio streaming, and the addition of new server endpoints that implement OpenAI’s real-time protocol.

The most significant change in this release is the introduction of a new mechanism for audio input and output. This enables more advanced multimodal use cases, such as real-time voice interaction and transcription.

Breaking Changes & Deprecations

This update includes changes to API design and CLI specifications. Developers and users utilizing these tools should be aware of the following changes:

Item Old Specification New Specification
CLI Arguments (May not be required) --model or -hf is mandatory
mtmd Audio Generation API Uses state_data buffer Uses mtmd_gen_state object
no_alloc Setting Parameter within llama_hparams Explicit argument during load/context creation

Changes to the CLI will affect users who launch models directly from the command line. Additionally, changes to the mtmd API will affect developers integrating audio generation features into their own applications. Regarding memory management changes, caution is required when handling low-level code that controls memory allocation.

Key Changes

Support for LiquidAI LFM3 Family and Many New Architectures

LiquidAI’s LFM3 series (text, vision, audio, omni) is now supported. In particular, LFM3-Audio-2B and LFM3-Omni-2B can process diverse inputs such as text, images, audio, and video, and feature ASR (Automatic Speech Recognition), TTS (Text-to-Speech), and voice chat capabilities. Furthermore, many new model architectures have been added, including DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, and Granite Switch Decision.

Introduction of Audio Streaming and Real-time APIs

As an expansion of multimodal functionality, an experimental mtmd_audio_stream API has been added to mtmd. This converts arriving PCM data into embedding rows available to the decoder. For server functionality, a /v1/realtime endpoint implementing OpenAI’s real-time protocol and a POST /tts endpoint for text-to-speech have been introduced. This enables real-time interaction and transcription using audio streaming I/O.

Memory Management and MoE Cache Optimization

The memory management mechanism has been revamped. no_alloc (the setting to perform no memory allocation) has been moved from model hyperparameters to an option during loading or context creation. This ensures that the no_alloc setting is properly respected in each memory cache, such as llama_kv_cache and llama_moe_cache. Additionally, for MoE (Mixture of Experts), the caching method for experts held in host memory has been changed from the traditional LRU to 50/50 SLRU. Furthermore, Sparse Attention (QSA/DSA) has been redesigned.

UI/UX Enhancements and Improved Model Management

The user interface has been significantly improved. A feature to search for and download models directly from the Hugging Face Hub has been added, making model management easier. Additionally, a dialog for searching past conversations and a “conversation compaction" feature to keep long conversations within context limits have been implemented. This makes local model usage and chat history management smoother.

Supported Models & Hardware

A very large number of new model architectures were supported in this update. The main supported models are as follows:

  • LiquidAI LFM3 Family: LFM3-1.7B (text), LFM3-VL-2B (vision), LFM3-Audio-2B, LFM3-Omni-2B. Specifically, the Audio and Omni models support audio streaming I/O, ASR, TTS, and voice chat.
  • LiquidAI Decision Models: d1-2B, d1-omni-2B.
  • Other New Architectures: DeepSeek V4.1 Flash, SenseNova U1, Beam (Reflection), Kolibri-1 (Aleph-Alpha), Flint, Muse Spark Open (text, vision and audio), Granite Switch Decision (256-way decision head), H2O-Lightning-4B, MiniCPM-OCR 1.0/Jev, Soprano TTS.
  • Embedding Models: Nemotron-3.5-Embed (supports non-standard GQA and BPE tokenizer).

Additionally, due to backend performance improvements, inference efficiency has been improved in specific hardware environments through the introduction of chunked graph launching in CUDA, graph replay in SYCL, Q2_K/Q3_K in OpenCL, Hexagon HTP reconstruction, and KleidiAI SVE kernels.

How to Get It

To install llama.cpp, please use the following commands depending on your environment:

# Example for updating the repository to the latest state and building
git pull && make

For detailed installation instructions, please check the official repository.

Speed on Our CPU

Measured by Local Model Watch on our CPU-only server (Neoverse-N1, 3 threads) with llama-bench (512-token prompt, 128-token generation), always on the same model: Qwen3-0.6B-Q4_K_M.gguf from unsloth/Qwen3-0.6B-GGUF (revision 50968a4468). Versions like v0.5.0 ship no binaries, so we used the first official ubuntu-arm64 build that contains the version; “+N” is how many commits that build is ahead of it. These are CPU figures, not a guide to GPU speed, and small differences can be run-to-run noise (± is the standard deviation).

Version Build measured Prompt Generation
v0.8.0 b11960 127.9 ± 0.1 tok/s 42.4 ± 0.4 tok/s
v0.7.0 b11669 128.4 ± 0.1 tok/s 43.1 ± 0.2 tok/s

Related Articles

What to Read Next

Sources