{"id":10175,"date":"2026-10-06T02:15:11","date_gmt":"2026-10-05T17:15:11","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/10\/06\/llamacpp-v0-6-0-released\/"},"modified":"2026-10-06T02:15:11","modified_gmt":"2026-10-05T17:15:11","slug":"llamacpp-v0-6-0-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/06\/llamacpp-v0-6-0-released\/","title":{"rendered":"llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nggml-org\/llama.cpp<\/a><\/td>\n<\/tr>\n
Version<\/td>\nv0.6.0<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-10-06<\/td>\n<\/tr>\n
License<\/td>\nMIT<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code when the information was collected. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

The latest version v0.6.0 of llama.cpp, the C++ LLM inference engine, has been released. llama.cpp is an open-source project developed under the MIT license, aimed at running large language models efficiently on consumer-grade hardware.<\/p>\n

The most impactful change in this update is the introduction of the new batch processing API, llama_batch_ext<\/code>. This enables the mixed processing of tokens and embeddings within a single batch, strengthening support for advanced inference techniques such as MTP (Multi-Token Prediction). For users developing custom tools using the API or those wanting to leverage the latest speculative decoding features, this is an important release to consider for migration.<\/p>\n

Breaking Changes and Deprecations<\/h2>\n

Internal session formats and API specifications have been updated, meaning previous configurations and some code will no longer work as-is.<\/p>\n

\n\n\n\n\n\n\n\n
Old Element<\/th>\nNew Element<\/th>\n<\/tr>\n<\/thead>\n
llama_batch<\/code><\/td>\nllama_batch_ext<\/code> (recommended)<\/td>\n<\/tr>\n
LLAMA_SESSION_VERSION<\/code> 10 or lower<\/td>\nLLAMA_SESSION_VERSION<\/code> 11<\/td>\n<\/tr>\n
LLAMA_STATE_SEQ_VERSION<\/code> 3 or lower<\/td>\nLLAMA_STATE_SEQ_VERSION<\/code> 4<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Because session and sequence state versions have been updated, session files and state data saved with previous versions may no longer be readable in this release. Additionally, developers are encouraged to migrate from the legacy llama_batch<\/code> API to the more flexible llama_batch_ext<\/code>.<\/p>\n

Key Changes<\/h2>\n

Introduction of the Extended Batch API llama_batch_ext<\/code><\/h3>\n

An improved API, llama_batch_ext<\/code>, has been implemented to increase inference flexibility. This API allows tokens and embeddings to be included in the same batch, as well as attaching “state” embeddings per token. This makes it easier to support specialized model structures like MTP and Deepstack. Developers embedding llama.cpp as a library will need to rewrite code for the new API, but gain the ability to build more complex inference pipelines.<\/p>\n

Major Speedups for the Metal Backend<\/h3>\n

Inference performance in Apple Silicon environments (Mac) has significantly improved. A new Flash Attention kernel for F16 KV caches has been added, alongside new matrix multiplication (mat-mul) kernels optimized for speculative decoding and batch processing. According to reports, matrix operations on Apple GPUs are accelerated by up to approximately 3x. Users running local LLMs on a Mac can expect to experience a noticeable speed boost after updating.<\/p>\n

Web UI Overhaul and Hugging Face Hub Integration<\/h3>\n

The UI provided by the built-in web server has been greatly enhanced. A Hugging Face Hub data layer has been integrated, implementing a pipeline to search for and download models directly from the browser. A feature to estimate whether a selected model fits into the current PC’s memory has also been added. Users who previously managed models via the command line can now experiment with models much more intuitively.<\/p>\n

Support for Qwen4Exp and MTP Speculative Decoding<\/h3>\n

Speculative decoding using MTP (Multi-Token Prediction) is now available for Qwen4Exp models. Tests in a DGX Spark environment reportedly show a decoding speed increase of about 1.5x. Alongside this, optimizations to halve the memory used for indexer score calculation and improvements to mask generation efficiency have been implemented. Using these features requires re-downloading corresponding models and checking configuration settings.<\/p>\n

Addition of Decision Model API \/v1\/systemone<\/code><\/h3>\n

A new endpoint, \/v1\/systemone<\/code>, has been added to the server features to handle Decision Models. Models such as Laya, Julia-1, Lev, OpenJev, and Kev are supported; these extend traditional embedding models tailored for specific decision-making tasks. OpenJev also supports image inputs, allowing multimodal decision-making tasks to be executed via the API.<\/p>\n

Update to Core Library ggml v0.26.0<\/h3>\n

The underlying numerical computation library ggml has been updated to v0.26.0. This update brings many low-level improvements, including added support for BF16 operations on the CPU backend, implementations of sparse Flash Attention in Vulkan and Metal, and build support for Windows ARM64 environments. Furthermore, GGUF format size validation has been made stricter, making the detection of corrupted model files more reliable.<\/p>\n

Supported Models and Hardware<\/h2>\n

Newly Supported Models<\/h3>\n

This update brings support for numerous new model architectures, allowing users to convert these state-of-the-art models into the GGUF format and run local inference.<\/p>\n