ggml v0.24.0 Released with Backend Improvements and API Updates

September 14, 2026September 18, 2026

At a Glance

Values determined by this site’s code at collection time. Dates are JST.

Overview

ggml-org/ggml v0.24.0 has been released. ggml is a machine learning tensor library (written in C++, MIT license, 15,352 stars) used as the foundation for many local LLM inference engines.

This version focuses on expanding backend coverage and robustness, introducing a new precision control API, major backend improvements for Vulkan, SYCL, Hexagon, and OpenCL, alongside numerous fixes and performance enhancements across various backends.

Key Changes

• Expanded the new precision control API ggml_prec with GGML_PREC_BF16, F16, Q8, and Q4 (GGML_PREC_DEFAULT is now deprecated)
• Added ggml_prec_set_acc() and ggml_prec_set_src() to control the precision of MUL_MAT, MUL_MAT_ID, and flash attention, deprecating the older ggml_mul_mat_set_prec() and ggml_flash_attn_ext_set_prec()
• Backend scheduler changes prevent unnecessary splitting of backend inputs and skip size-0 MoE ids tensors
• Added PCH (Precompiled Header) and unity-build support to reduce build times

Supported Models and Hardware

• CPU: Added repacked support for q4_0 and Q1_0 vector intrinsics on s390x, guarded VXE-specific helpers, and added non-VXE tests
• CUDA / HIP / MUSA: Added dedicated MMQ configuration tables for AMD GCN and gfx90c HIP support. Devices without hardware acceleration for BF16 now fall back to F32, and flash attention tuning was added for gfx1201
• Metal: Added vector tuning for flash attention on M2 Max and M3
• Vulkan: Added support for TQ1_0 and iq4_xs, type-aligned GET_ROWS, f16 B-type matrix multiplication pipelines, and DeepSeek-V4 hyper-connection fused operations
• OpenCL: Added Adreno xmem SDPA path and A8 q4_0/q4_K binary GEMM kernels
• SYCL: Added radix-select for top_k, batched L2_NORM, and IQ type handling for MoE
• Hexagon: Added multi-device model splitting (row-split) support, along with RELU and LEAKY_RELU operations
• WebGPU: Added Dawn updates and type-aligned GET_ROWS support

How to Get It

To build from source, run the following commands:

git clone https://github.com/ggml-org/ggml
cd ggml

mkdir build && cd build
cmake..
cmake --build. --config Release -j 8

Releases Since Our Last Article

Compiled by Local Model Watch from the project’s GitHub releases: the versions between this release and the last one we covered, which did not get separate articles. Full history: release tracker.

Sources

• https://github.com/ggml-org/ggml/releases/tag/v0.24.0