llama.cpp v0.5.0 Released with Backend and Server Upgrades

llama.cpp v0.5.0 Released with Backend and Server Upgrades

At a Glance

Item Value
Repository ggml-org/llama.cpp
Version v0.5.0
Published 2026-09-24
License MIT
Source type Primary source (the publisher itself)

Values determined by this site’s code at collection time. Dates are JST.

Overview

v0.5.0, the latest version of the C/C++ LLM inference library llama.cpp, has been released. This update focuses on improving backend performance, expanding model support, and enhancing the robustness of server and router operations.

The most impactful changes for users are hardware acceleration enhancements accompanying backend (ggml) updates, such as accelerated conv2d via implicit GEMM in CUDA and fused optimization for MoE and SSM_CONV in Metal. These are expected to improve inference performance under specific environments.

Key Changes

Backend and Hardware Acceleration Optimization

In CUDA environments, conv2d operations have been accelerated using implicit GEMM. Additionally, in Metal environments, fusion optimization for MoE (Mixture of Experts) and SSM_CONV has been added. These changes directly affect inference speeds for users utilizing NVIDIA GPUs or Apple Silicon.

Server Feature Expansion and Multi-Address Support

In llama-server, the --host option can now accept comma-separated TCP addresses and UNIX sockets. This enables binding to multiple network interfaces simultaneously—for example, listening to connections via VPN and localhost at the same time. This is a very useful feature for engineers operating servers across multiple network environments.

Support for New Model Architectures

Support for numerous new models and architectures has progressed. Specifically, this includes support for HRM-Text (DFM Mimir 1B), conversion support for MiMo-V2.6, and DFlash (speculative decoding) support for HunyuanOCR. Furthermore, expanded support for Nemotron MTP and implementation of hyper-connection operations and sparse flash attention for Qwen4Exp have also been carried out. Users wanting to try new models locally can now use them in GGUF format thanks to these additions.

Multimodal Support in Server Output

Support for input_image has been added to server function-calling outputs. This enables the construction of more advanced multimodal agent functions, such as handling image data as the result of tool execution. This affects developers utilizing multimodal models like Gemma 4 to process complex tasks involving images.

Sampling Control via Environment Variables

Settings for temperature, top-p, min-p, and various penalties can now be configured via environment variables. This allows users to control sampling parameters simply by setting execution environment configurations without rewriting code. Convenience is improved for users who want to dynamically adjust inference behavior.

Supported Models and Hardware

This release enhances support for the following model architectures and formats:

  • HRM-Text (DFM Mimir 1B): Newly supported. This model features a structure that alternately executes two transformer stacks and is available in GGUF format.
  • MiMo-V2.6 (Pro/Flash): Conversion support has been added. These models use mxfp4 experts and can be loaded into llama.cpp while retaining vision capabilities.
  • HunyuanOCR: Speculative decoding using DFlash is supported. It is reported that this improves decoding speed by approximately 2x in document image OCR processing.
  • Nemotron Series: MTP (Multi-Token Prediction) support has been expanded in Nemotron Super 3.
  • Qwen4Exp: Hyper-connection operations and sparse flash attention support have been implemented.
  • Muse Glimmer: Support has been added via the --fuse-qkv option.
  • Gemma 4: Parsing fixes have been applied, including support for the Gemma 4 DSpark draft backbone.

How to Get It

To update llama.cpp, use one of the following methods:

Building from Source Clone the repository and rebuild according to the build instructions.

git pull && make

Using Docker If you are using Docker images, pull the latest image.

Using Binaries Download the latest prebuilt binaries from the releases page.

Related Articles

Sources

Engines and ToolsApple Silicon,CUDA,ggml,GGUF,llama.cpp,Verified