{"id":13980,"date":"2026-11-17T11:02:50","date_gmt":"2026-11-17T02:02:50","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=13980"},"modified":"2026-11-17T11:12:05","modified_gmt":"2026-11-17T02:12:05","slug":"llama-cpp-v0-8-0-multimodal-lfm3-support","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/llama-cpp-v0-8-0-multimodal-lfm3-support\/","title":{"rendered":"llama.cpp v0.8.0: Multimodal Enhancements and LFM3 Support"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\">ggml-org\/llama.cpp<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/releases\/tag\/v0.8.0\">v0.8.0<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-28<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>MIT<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p><code>llama.cpp<\/code> is an LLM (Large Language Model) inference engine written in C\/C++. In this v0.8.0 update, multimodal capabilities have been significantly enhanced, including support for LiquidAI&#8217;s LFM3 family, new APIs for audio streaming, and the addition of new server endpoints that implement OpenAI&#8217;s real-time protocol.<\/p>\n<p>The most significant change in this release is the introduction of a new mechanism for audio input and output. This enables more advanced multimodal use cases, such as real-time voice interaction and transcription.<\/p>\n<h2>Breaking Changes &amp; Deprecations<\/h2>\n<p>This update includes changes to API design and CLI specifications. Developers and users utilizing these tools should be aware of the following changes:<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th style=\"text-align: left;\">Item<\/th>\n<th style=\"text-align: left;\">Old Specification<\/th>\n<th style=\"text-align: left;\">New Specification<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: left;\">CLI Arguments<\/td>\n<td style=\"text-align: left;\">(May not be required)<\/td>\n<td style=\"text-align: left;\"><code>--model<\/code> or <code>-hf<\/code> is mandatory<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\"><code>mtmd<\/code> Audio Generation API<\/td>\n<td style=\"text-align: left;\">Uses <code>state_data<\/code> buffer<\/td>\n<td style=\"text-align: left;\">Uses <code>mtmd_gen_state<\/code> object<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\"><code>no_alloc<\/code> Setting<\/td>\n<td style=\"text-align: left;\">Parameter within <code>llama_hparams<\/code><\/td>\n<td style=\"text-align: left;\">Explicit argument during load\/context creation<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Changes to the CLI will affect users who launch models directly from the command line. Additionally, changes to the <code>mtmd<\/code> API will affect developers integrating audio generation features into their own applications. Regarding memory management changes, caution is required when handling low-level code that controls memory allocation.<\/p>\n<h2>Key Changes<\/h2>\n<h3>Support for LiquidAI LFM3 Family and Many New Architectures<\/h3>\n<p>LiquidAI&#8217;s LFM3 series (text, vision, audio, omni) is now supported. In particular, LFM3-Audio-2B and LFM3-Omni-2B can process diverse inputs such as text, images, audio, and video, and feature ASR (Automatic Speech Recognition), TTS (Text-to-Speech), and voice chat capabilities. Furthermore, many new model architectures have been added, including DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, and Granite Switch Decision.<\/p>\n<h3>Introduction of Audio Streaming and Real-time APIs<\/h3>\n<p>As an expansion of multimodal functionality, an experimental <code>mtmd_audio_stream<\/code> API has been added to <code>mtmd<\/code>. This converts arriving PCM data into embedding rows available to the decoder. For server functionality, a <code>\/v1\/realtime<\/code> endpoint implementing OpenAI&#8217;s real-time protocol and a <code>POST \/tts<\/code> endpoint for text-to-speech have been introduced. This enables real-time interaction and transcription using audio streaming I\/O.<\/p>\n<h3>Memory Management and MoE Cache Optimization<\/h3>\n<p>The memory management mechanism has been revamped. <code>no_alloc<\/code> (the setting to perform no memory allocation) has been moved from model hyperparameters to an option during loading or context creation. This ensures that the <code>no_alloc<\/code> setting is properly respected in each memory cache, such as <code>llama_kv_cache<\/code> and <code>llama_moe_cache<\/code>. Additionally, for MoE (Mixture of Experts), the caching method for experts held in host memory has been changed from the traditional LRU to 50\/50 SLRU. Furthermore, Sparse Attention (QSA\/DSA) has been redesigned.<\/p>\n<h3>UI\/UX Enhancements and Improved Model Management<\/h3>\n<p>The user interface has been significantly improved. A feature to search for and download models directly from the Hugging Face Hub has been added, making model management easier. Additionally, a dialog for searching past conversations and a &#8220;conversation compaction&#8221; feature to keep long conversations within context limits have been implemented. This makes local model usage and chat history management smoother.<\/p>\n<h2>Supported Models &amp; Hardware<\/h2>\n<p>A very large number of new model architectures were supported in this update. The main supported models are as follows:<\/p>\n<ul>\n<li><strong>LiquidAI LFM3 Family<\/strong>: LFM3-1.7B (text), LFM3-VL-2B (vision), LFM3-Audio-2B, LFM3-Omni-2B. Specifically, the Audio and Omni models support audio streaming I\/O, ASR, TTS, and voice chat.<\/li>\n<li><strong>LiquidAI Decision Models<\/strong>: d1-2B, d1-omni-2B.<\/li>\n<li><strong>Other New Architectures<\/strong>: DeepSeek V4.1 Flash, SenseNova U1, Beam (Reflection), Kolibri-1 (Aleph-Alpha), Flint, Muse Spark Open (text, vision and audio), Granite Switch Decision (256-way decision head), H2O-Lightning-4B, MiniCPM-OCR 1.0\/Jev, Soprano TTS.<\/li>\n<li><strong>Embedding Models<\/strong>: Nemotron-3.5-Embed (supports non-standard GQA and BPE tokenizer).<\/li>\n<\/ul>\n<p>Additionally, due to backend performance improvements, inference efficiency has been improved in specific hardware environments through the introduction of chunked graph launching in CUDA, graph replay in SYCL, Q2_K\/Q3_K in OpenCL, Hexagon HTP reconstruction, and KleidiAI SVE kernels.<\/p>\n<h2>How to Get It<\/h2>\n<p>To install <code>llama.cpp<\/code>, please use the following commands depending on your environment:<\/p>\n<pre><code class=\"language-bash\"># Example for updating the repository to the latest state and building\ngit pull &amp;&amp; make\n<\/code><\/pre>\n<p>For detailed installation instructions, please check the official repository.<\/p>\n<p><!-- lmw:version-speed --><\/p>\n<h2>Speed on Our CPU<\/h2>\n<p><em>Measured by Local Model Watch on our CPU-only server (Neoverse-N1, 3 threads) with llama-bench (512-token prompt, 128-token generation), always on the same model: <code>Qwen3-0.6B-Q4_K_M.gguf<\/code> from <a href=\"https:\/\/huggingface.co\/unsloth\/Qwen3-0.6B-GGUF\/tree\/50968a4468ef4233ed78cd7c3de230dd1d61a56b\">unsloth\/Qwen3-0.6B-GGUF<\/a> (revision <code>50968a4468<\/code>). Versions like v0.5.0 ship no binaries, so we used the first official ubuntu-arm64 build that contains the version; \u201c+N\u201d is how many commits that build is ahead of it. These are CPU figures, not a guide to GPU speed, and small differences can be run-to-run noise (\u00b1 is the standard deviation).<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Version<\/th>\n<th>Build measured<\/th>\n<th>Prompt<\/th>\n<th>Generation<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>v0.8.0<\/td>\n<td>b11960<\/td>\n<td>127.9 \u00b1 0.1 tok\/s<\/td>\n<td>42.4 \u00b1 0.4 tok\/s<\/td>\n<\/tr>\n<tr>\n<td>v0.7.0<\/td>\n<td>b11669<\/td>\n<td>128.4 \u00b1 0.1 tok\/s<\/td>\n<td>43.1 \u00b1 0.2 tok\/s<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:version-speed --><\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/06\/llamacpp-v0-6-0-released\/\">llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/24\/llama-cpp-v0-5-0-released\/\">llama.cpp v0.5.0 Released with Backend and Server Upgrades<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/llamacpp-v100-released\/\">llama.cpp v1.0.0 Released: Major Updates and New Features<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/llama-cpp-v0-9-0-released\/\">llama.cpp v0.9.0 Released: FP8 Quantization and More<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Follow this tool<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp overview and release history (9 releases tracked)<\/a><\/li>\n<li><strong>Other inference engines and runtimes<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-vllm-en\/\">vLLM<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-sglang-en\/\">SGLang<\/a><\/li>\n<li><strong>Engines mentioned in this article<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ggml-en\/\">ggml<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/releases\/tag\/v0.8.0\">ggml-org\/llama.cpp v0.8.0 (GitHub Release)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/31252\">PR #31252: LFM3 family support<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/31257\">PR #31257: LFM3-Audio-2B and LFM3-Omni-2B support<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/31087\">PR #31087: respect no_alloc in all memory caches<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/31141\">PR #31141: move no_alloc out of llama_hparams<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/31200\">PR #31200: remove obsolete n_pad from memory instances<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/31035\">PR #31035: fix exhausting CPU and MEM caused by dry sequence breaker<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30629\">PR #30629: mtmd: add mtmd_audio_stream + support voxtral-realtime<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30902\">PR #30902: mtmd: refactor mtmd gen API, add state object<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/28607\">PR #28607: mtmd : add Soprano audio generation<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>llama.cpp v0.8.0 introduces LiquidAI LFM3 support, real-time audio streaming APIs, and significant memory management optimizations for multimodal workflows<\/p>\n","protected":false},"author":1,"featured_media":13979,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[161,163,4177,2278,1073,1547],"class_list":["post-13980","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-ggml-en","tag-gguf-en","tag-lfm3-en","tag-liquidai-en","tag-llama-cpp-en","tag-verified"],"lang":"en","translations":{"en":13980,"ja":13978},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13980","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=13980"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13980\/revisions"}],"predecessor-version":[{"id":17980,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13980\/revisions\/17980"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/13979"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=13980"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=13980"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=13980"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}