llama.cpp v1.0.0 Released: Major Updates and New Features

llama.cpp v1.0.0 Released: Major Updates and New Features

At a Glance

Item Value
Repository ggml-org/llama.cpp
Version v1.0.0
Published 2026-11-13
License MIT
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

llama.cpp is an LLM inference engine written in C/C++ (MIT license). Version 1.0.0 has been released. The official note simply states “More info at 3PM CET", and detailed explanations are expected to be added later. The following is based on the commit list since v0.9.0, and details remain unconfirmed.

The change with the most impact for users is the security hardening of llama-server. It includes fixes for --cors-origins processing and DNS rebinding mitigation (#32098), as well as SSRF fixes for CORS proxies and media URLs (#32115). Users exposing servers to LANs or other networks should consider updating promptly.

Key Changes

Commit titles are the only basis, as PR descriptions are not available in the source material. Therefore, specific details of behaviors cannot be definitively confirmed.

New Features

FP8 KV-cache and CUDA OCP FP8 Support

llama : Add FP8 KV-cache support(#28392) and CUDA: Add OCP FP8 support(#28413) have been added. This targets CUDA users looking to save VRAM. Usage instructions and supported GPUs are not in the documentation, pending official follow-up.

Windows Unbuffered Model Load

Windows unbuffered model load(#26014) has been added. This is relevant for users loading large models on Windows. Instructions on how to enable it are not documented.

External PLE File Loading

feat: external PLE file loading(#31185) has been added. Details are not provided in the documentation.

Server Audio and MCP Related

  • OpenAI-compatible /v1/audio/speech endpoint has been added(#31763)
  • MCP tool calls are now cancelled on client disconnection or timeout(#32025)
  • When --tools-runtime is specified, x-tool-runtime is ignored(#32119)

MoE Cache Improvements

An LRFU/MQ-style hybrid caching method(#31233) has been introduced. Processing to execute experts not present in the cache on the CPU(#30966) has also been added. Users running MoE models that do not fit entirely in VRAM are advised to verify the behavior.

Performance Improvements

  • CUDA: Few-row MMQ for batch sizes 2–24 (#31533)
  • CUDA: Optimized nvfp4 checkpoint prefill with scale fusion(#26048)
  • Vulkan: FlashAttention kernel for prefill targeting Intel(#29357)
  • ARM NEON: Utilization of 4×4 simd_gemm tiles(#32128)
  • Hexagon: Support for Q5_0 and Q5_1(#32094)

Specific numerical performance gains are not included in the documentation.

Bug Fixes and Security

  • A change has been introduced where llama-server treats tool usage on non-loopback addresses as requiring explicit opt-in(#32113). In externally exposed environments, the behavior of tool functions may change (details unconfirmed)
  • Restricted Hugging Face tokens to be sent only to the huggingface.co host(#31730)
  • Fixed an issue where llama-cli would crash on invalid UTF-8 in prompts or files(#31563)
  • Fixed out-of-bounds writes to Metal FlashAttention output buffers(#31662)
  • Fixed an issue in llama-quantize where imatrix GGUF file paths leaked into metadata(#32000)
  • In mtmd, strict validation of mandatory tensors was added for projectors such as Qwen-VL and HunyuanVL(#32129, #32130, #32131). If your existing mmproj fails to load, try re-converting or re-acquiring it (unconfirmed)

Supported Models and Hardware

This version brings support for multiple new model architectures, various accelerators, and quantization formats.

Newly Supported Models and Extensions

According to the documentation’s commit list, the following model architectures and feature additions have been reported:

  • Kimi-K3.1: Support added for the text model Kimi-K3.1-Pro(#31779) and kimi K3.1 vision(#31980).
  • Gemma 4: In addition to schema rule organization(#31915), support for input audio streams(#31965) has been added.
  • TTS (Text-to-Speech) Models: Support and feature extensions implemented for drama-3 TTS(#31613) and Pocket TTS(#31655).
  • Other Models: Added support for models such as Ripple (ricursiveai)(#31692), xing4_0(#31589), and MageVL-Cowork(#31819). Byte fallback token processing fixes for Sarvam(#32006) are also included.

In multimodal (mtmd) processing, mandatory F32 positional encoding has been introduced for HunyuanVL(#32131), along with improved projection width loading for MiniCPM-V(#32132) and the requirement for 2×2 spatial merging in the Qwen-VL projector(#32130), streamlining inference processing for related models.

Quantization and Accelerator Support

Optimizations and expansions have also been carried out across many backends for hardware accelerators and quantization formats.

  • Hexagon: Added support for Q5_0 and Q5_1 quantization formats(#32094).
  • CUDA / GPU: Implemented OCP FP8 format support(#28413), scale fusion optimization for nvfp4 checkpoints(#26048), and MFMA indexer for CDNA2 (gfx90a)(#29050). Weight scale attachment for Granite NVFP4(#31449) is also supported.
  • Vulkan: Added FlashAttention kernel for prefill targeting Intel(#29357) and Large-Tile Split-K optimizations for AMD RDNA3/4(#30280).
  • OpenCL / Mobile: Added decomposed FlashAttention for Qualcomm Adreno(#32046), and multi-column GEMV for F32 and Q5_K on Adreno X2 enabled by default(#31694).
  • SpacemiT / RISC-V: Added GATED_DELTA_NET routing to SpacemiT RVV kernels(#31897), SpacemiT Q4_1 32×256 HP kernel dispatch(#31929), and enabling of SpacemiT IME2 MXFP4 Dense Repack(#31776).
  • SYCL / WebGPU: Added device-side MoE scheduler in the SYCL backend(#30156), and implemented PAD_REFLECT_1D(#31877), COUNT_EQUAL(#31578), MEAN(#31591), and Group Norm kernels(#31490) in WebGPU.

How to Get It

The documentation’s build guide does not list specific installation commands. For environment-specific instructions on obtaining the latest binaries or update procedures, please refer directly to the llama.cpp releases page.

If updating from source code, fetch the latest code using standard git commands and rebuild:

git pull
cmake -B build
cmake --build build --config Release

Speed on Our CPU

Measured by Local Model Watch on our CPU-only server (Neoverse-N1, 3 threads) with llama-bench (512-token prompt, 128-token generation), always on the same model: Qwen3-0.6B-Q4_K_M.gguf from unsloth/Qwen3-0.6B-GGUF (revision 50968a4468). Versions like v0.5.0 ship no binaries, so we used the first official ubuntu-arm64 build that contains the version; “+N” is how many commits that build is ahead of it. These are CPU figures, not a guide to GPU speed, and small differences can be run-to-run noise (± is the standard deviation).

Version Build measured Prompt Generation
v1.0.0 b12359 132.7 ± 0.6 tok/s 42.5 ± 0.7 tok/s
v0.9.0 b12110 128.1 ± 0.9 tok/s 42.9 ± 0.6 tok/s

Related Articles

What to Read Next

Sources