ggml v0.24.0 Released with Backend Improvements and API Updates

October 28, 2026

ggml v0.24.0 Released with Backend Improvements and API Updates

At a Glance

Item Value
Repository ggml-org/ggml
Version v0.24.0
Published 2026-09-14
License MIT
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

ggml-org/ggml v0.24.0 has been released. ggml is a machine learning tensor library (written in C++, MIT license) used as the foundation for many local LLM inference engines.

This version focuses on expanding backend coverage and robustness, introducing a new precision control API, major backend improvements for Vulkan, SYCL, Hexagon, and OpenCL, alongside numerous fixes and performance enhancements across various backends.

Key Changes

  • Expanded the new precision control API ggml_prec with GGML_PREC_BF16, F16, Q8, and Q4 (GGML_PREC_DEFAULT is now deprecated)
  • Added ggml_prec_set_acc() and ggml_prec_set_src() to control the precision of MUL_MAT, MUL_MAT_ID, and flash attention, deprecating the older ggml_mul_mat_set_prec() and ggml_flash_attn_ext_set_prec()
  • Backend scheduler changes prevent unnecessary splitting of backend inputs and skip size-0 MoE ids tensors
  • Added PCH (Precompiled Header) and unity-build support to reduce build times

Supported Models and Hardware

  • CPU: Added repacked support for q4_0 and Q1_0 vector intrinsics on s390x, guarded VXE-specific helpers, and added non-VXE tests
  • CUDA / HIP / MUSA: Added dedicated MMQ configuration tables for AMD GCN and gfx90c HIP support. Devices without hardware acceleration for BF16 now fall back to F32, and flash attention tuning was added for gfx1201
  • Metal: Added vector tuning for flash attention on M2 Max and M3
  • Vulkan: Added support for TQ1_0 and iq4_xs, type-aligned GET_ROWS, f16 B-type matrix multiplication pipelines, and DeepSeek-V4 hyper-connection fused operations
  • OpenCL: Added Adreno xmem SDPA path and A8 q4_0/q4_K binary GEMM kernels
  • SYCL: Added radix-select for top_k, batched L2_NORM, and IQ type handling for MoE
  • Hexagon: Added multi-device model splitting (row-split) support, along with RELU and LEAKY_RELU operations
  • WebGPU: Added Dawn updates and type-aligned GET_ROWS support

How to Get It

To build from source, run the following commands:

git clone https://github.com/ggml-org/ggml
cd ggml

mkdir build && cd build
cmake..
cmake --build. --config Release -j 8

Related Articles

What to Read Next

Sources

Engines and ToolsCUDA,ggml,GGUF,LLM,Metal,Vulkan,確認済み