{"id":10033,"date":"2026-10-06T00:29:53","date_gmt":"2026-10-05T15:29:53","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/10\/06\/ggml-v0260-released\/"},"modified":"2026-10-06T02:16:23","modified_gmt":"2026-10-05T17:16:23","slug":"ggml-v0260-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/06\/ggml-v0260-released\/","title":{"rendered":"ggml v0.26.0 Released with Sparse Flash Attention & New Backends"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nggml-org\/ggml<\/a><\/td>\n<\/tr>\n
Version<\/td>\nv0.26.0<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-10-05<\/td>\n<\/tr>\n
License<\/td>\nMIT<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code when the information was collected. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

ggml-org\/ggml is a C++ tensor library (MIT license) that serves as the foundation for inference engines like llama.cpp. It handles computation graph construction and execution for machine learning models as well as backend abstraction, forming the base for llama.cpp and many tools that use it. The newly released v0.26.0 compiles changes made since v0.25.3.<\/p>\n

The most widespread change in this release is the addition and optimization of sparse Flash Attention kernels for the SYCL, Vulkan, and Metal backends. Sparse FA for quantized KV caches has been newly implemented, which relates to memory usage and speed when handling long contexts. Users running llama.cpp-based tools on Intel GPUs (SYCL), AMD\/Intel Vulkan environments, or Apple Silicon (Metal) are likely to benefit from this update.<\/p>\n

Key Changes<\/h2>\n

Lightning Indexer Memory Halving and Tiling Optimization<\/strong><\/p>\n

On CUDA, Metal, and Vulkan, optimizations have been added to halve the score memory for the lightning indexer and perform tiling for keys and tokens. In addition, the MUSA (Moore Threads) backend now uses vector-version lightning indexer kernels. Improvements in memory usage and speed can be expected when running models that utilize the indexer.<\/p>\n

CPU Backend BF16 Support and Tile mul_mat for k-quants<\/strong><\/p>\n

BF16 unary\/GLU\/binary\/scale operations have been added to the CPU side, and BF16 is now accepted as src1<\/code> for mul_mat<\/code>. Furthermore, tile mul_mat<\/code> for k-quants using int8 unpack tiles and 16×16 micro-kernels has been introduced. This change is relevant for users who do not have a GPU and run BF16 models or k-quant (such as Q3_K\/Q4_K\/Q5_K\/Q6_K) quantized models using only the CPU.<\/p>\n

CUDA W4A4 Paths for NVFP4\/MXFP4 and Shared-Expert Fusion in MMVQ<\/strong><\/p>\n

A mul_mat<\/code> path for W4A4 (NVFP4\/MXFP4), controllable from the model side via llama_prec_policy<\/code>, has been added. Calculation type handling in the NVFP4 MMQ side and cuBLAS paths has also been optimized. Additionally, optimizations to fuse MoE model shared experts into MMVQ have been included. This is relevant when running NVFP4\/MXFP4 format models or MoE models on NVIDIA GPUs.<\/p>\n

Hexagon Backend Quantization Support and Sampler Addition<\/strong><\/p>\n

Sampler functionality has been added to the Hexagon (Snapdragon NPU) backend, and q2_k, q3_k, and q5_k have been added to the supported quantization types. Dynamic quantizer improvements have also been made. This is relevant for users running models using the Hexagon backend on Snapdragon-equipped devices.<\/p>\n

Faster Model Loading and Enhanced GGUF Validation<\/strong><\/p>\n

In addition to faster model loading, a bug where loading invalid GGUF files with extremely large KV dimensions caused a hang has been fixed. Furthermore, GGUF size validation has been tightened to detect and reject integer overflows and tensor sizes that wrap after padding. This change relates to stability and safety when handling GGUF files from unknown sources.<\/p>\n

Windows ARM64 (MSVC) Build Enabled<\/strong><\/p>\n

Building with MSVC’s cl.exe<\/code> has been enabled in Windows ARM64 environments. This is relevant for users who want to perform native builds on Snapdragon-powered Windows PCs and other similar devices.<\/p>\n

Supported Models and Hardware<\/h2>\n

This version advances support for new architectures and quantization formats across a wide variety of backends. It includes changes that directly translate to performance improvements and expanded runnable models for users operating in specific hardware environments.<\/p>\n

Expanded GPU and Accelerator Support<\/h3>\n

In the Vulkan backend, matrix multiplication (matmul) using int8 coopmat1<\/code> has been implemented for AMD’s RDNA3 and RDNA4 architectures. This is expected to improve inference efficiency on the latest Radeon series and Ryzen integrated GPUs. Additionally, fine-grained adjustments have been made per device, such as tuning GDN kernels for Intel GPUs, adjusting tile sizes for Samsung GPUs with 32KB shared memory, and optimizing argmax kernel selection for Adreno GPUs.<\/p>\n

In Apple Silicon (Metal) environments, Flash Attention kernels for F16 format KV caches have been added. Furthermore, calculations at BF16 precision are now supported in MXFP4 format matrix operations, aiming to balance precision retention and speed. Optimizations for sparse Flash Attention are also progressing.<\/p>\n

For the SYCL backend targeting Intel GPUs, in addition to sparse Flash Attention support, wide loads for DMMV ESIMD and MMVQ in Q8_0 format are now supported. This improves execution speed for quantized models on Intel Arc and data center GPUs. Moreover, tensor AllReduce synchronization overhead has been reduced by utilizing pinned host buffers.<\/p>\n

Enhanced WebGPU, Mobile, and Edge Environments<\/h3>\n

The WebGPU backend, running inside browsers, has significantly increased its supported quantization formats. Q1_0, Q5_0, Q5_1, Q3_K, Q5_K, Q6_K, and MXFP4 MMVQ (Matrix-Matrix Vector Quantization) are now supported. Furthermore, bfloat16 format matrix operations and row fetching (GET_ROWS<\/code>) are now possible, greatly expanding local inference options in web browser environments.<\/p>\n

For Qualcomm’s Snapdragon Hexagon NPU, new quantization types such as q2_k, q3_k, and q5_k have been added. Combined with sampler feature support and dynamic quantizer improvements, power-efficient and fast inference on mobile devices becomes available for a wider range of models. The OpenCL backend also received fixes for the Q5_K gemm_nonshuffle<\/code> kernel targeting Adreno.<\/p>\n

Enterprise and Special Architectures<\/h3>\n

The OpenVINO backend has been updated to version 2026.4.1. This brings performance optimizations, an expanded set of supported operations (ops), and improved device listing displays. It also includes optimizations for handling GET_ROWS<\/code> from weight views.<\/p>\n

In IBM’s z Systems (zDNN) backend, buffer reset implementations, memory leak fixes, and crash fixes related to zero-row tensors have been implemented, improving stability in mainframe environments. Furthermore, support for a wide range of architectures from edge to enterprise has been strengthened, such as adding the Q8_0 IME1 matrix kernel for SpacemiT X60 (RISC-V). The RPC backend received improvements using RDMA completion channels to avoid spinning.<\/p>\n

How to Get It<\/h2>\n

If you are building and using ggml from source, you can compile the latest version by following these steps according to the quick start guide:<\/p>\n

git clone https:\/\/github.com\/ggml-org\/ggml\ncd ggml\n\nmkdir build && cd build\ncmake..\ncmake --build. --config Release -j 8\n<\/code><\/pre>\n
To update an existing environment, run git pull<\/code> within the repository and then re-build using the steps above. When enabling specific backends (CUDA, Vulkan, Metal, etc.), appropriate flags must be passed during cmake..<\/code>. Check the project documentation for specific build flags for each backend.<\/p>\n
<\/p>\n
Releases Since Our Last Article<\/h2>\n
Compiled by Local Model Watch from the project’s GitHub releases: the versions between this release and the last one we covered, which did not get separate articles.<\/em> Full history: release tracker<\/a>.<\/em><\/p>\n
\n\n\n\n\n\n
Version<\/th>\nReleased<\/th>\nRelease notes<\/th>\n<\/tr>\n<\/thead>\n
v0.25.3<\/td>\n2026-09-25<\/td>\nGitHub<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n
<\/p>\n
<\/p>\n
Related Articles<\/h2>\n