ggml v1.0.0 Released: Major Version Update and Improvements

ggml v1.0.0 Released: Major Version Update and Improvements

At a Glance

Item Value
Repository ggml-org/ggml
Version v1.0.0
Published 2026-11-13
License MIT
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

Major version “v1.0.0" has been released for ggml-org/ggml, a C++ tensor library designed for machine learning. Available under the MIT license, ggml is widely used as a foundational library for efficient model inference across various hardware.

The most significant change in this version is the cleanup and complete removal of previously deprecated public APIs and legacy DIAG_MASK operators. Additionally, the RPC protocol has been updated to reject connections from incompatible peers. Users relying on older code or distributed inference environments tied to the old specification should verify compatibility before updating.

Breaking Changes & Deprecations

This release removes deprecated legacy APIs and past compatibility-maintaining operators.

Feature / Option Before Treatment / Migration Path
DIAG_MASK_INF / DIAG_MASK_ZERO operators Removed (migrate to new standard operators)
Deprecated public APIs Removed
RPC protocol (old version) Promoted to v9 (incompatible peer connections rejected)

These changes impact application developers who directly call the removed DIAG_MASK operators or old APIs, as well as users performing network-distributed inference using RPC. When using RPC features, ensure all connecting peers are updated to the v1.0.0 equivalent version.

Key Changes

Scheduler and Memory Transfer Optimizations

Discussions and improvements to the core backend scheduler skip uploading unread input tensors. The specification has also been optimized to prefetch only inputs generated by the previous split (llama/31419, llama/32100). Furthermore, for Mixture of Experts (MoE) caches, a feature was added to offload a portion of cache-miss calculations to the CPU side and asynchronously copy the results back (llama/30966). This prevents unnecessary data transfers and improves overall responsiveness during inference.

CUDA Backend Feature Expansion and Performance Improvements

For the CUDA backend targeting NVIDIA GPUs, optimizations for low-row MMQ (Matrix-Vector / Matrix Quantized) processing in batch sizes 2 to 24 and MMVF kernels for BF16 were added (llama/31533, llama/31959). Support for the OCP FP8 format was introduced alongside NVFP4 gate/up matrix multiplication kernels fused with GLU within MMQ (llama/28413, llama/31296). FlashAttention support also expanded to include head size 256 settings for sm70 and MMA head sizes 40/72 (llama/27997, llama/30285).

Enhancements to Metal, Vulkan, SYCL, and Other Backends

Optimizations across various GPU and accelerator environments have been broadly applied. The Metal backend fused MoE routing for DeepSeek-V4/V4.1 and GLM, applying speedups that keep fusible node groups within a single command buffer (llama/31615, llama/31688). Vulkan received Intel FlashAttention prefetch kernel additions and large-tile split-K optimizations for RDNA3/4 (llama/29357, llama/30280), while SYCL introduced a new device-side MoE scheduler (llama/30156). Decode speeds in mobile and edge environments have also been targeted, such as the addition of decoupled FlashAttention for Adreno in OpenCL environments (llama/32046).

YaRN Truncation Settings and Integration Across Backends

To flexibly adjust rounding behavior for YaRN scaling used in long-context processing, ggml_rope_set_truncate and ggml_rope_get_truncate were introduced (llama/30206). Support for this truncation setting has been uniformly expanded across multiple major backends, including OpenVINO, SYCL, Vulkan, WebGPU, and RPC (llama/30802), improving stability and control when running long-context models across a wide range of hardware.

Supported Models and Hardware

This release brings a wide range of added kernels and optimizations for various accelerators and CPU instruction sets, alongside support for new data precisions and quantization formats. Depending on the hardware environment used, users can expect improved inference speeds and expanded model support.

Extended Support for Quantization Formats and New Precisions

The OCP FP8 format and FP8 KV cache are now newly supported in the CUDA backend for NVIDIA GPUs (llama/28413, llama/28392). Q5_0 and Q5_1 quantization formats have been added for Hexagon DSPs (llama/32094). The OpenCL backend introduced q6_k tiled, q8_0, and f32/q5_k multi-column GEMV kernels, enabling them by default on Adreno X2 environments (llama/31758, llama/31694). On the CPU side, SpacemiT’s Q4_1 32×256 HP kernel, IME2 MXFP4 repack, and bf16/q8_0 VXE implementations in llamafile were added (llama/31929, llama/31776, llama/31866).

Optimizations for GPUs and Accelerators

Optimizations for individual GPU architectures and dedicated accelerators are promoted as follows:

  • NVIDIA GPU (CUDA / ROCm): FlashAttention added head size 256 settings for the SM70 architecture and MMA head sizes 40/72 support (llama/27997, llama/30285). Processing efficiency in ROCm/HIP environments was also improved via the introduction of the MFMA lightning indexer for AMD CDNA2 (gfx90a) and the adoption of fixed BLAS workspaces (llama/29050, llama/31838).
  • Apple Silicon (Metal): Added support for MoE routing fusion in DeepSeek-V4/V4.1 and GLM (llama/31615). Control overhead is reduced by holding fusible node groups within a single command buffer (llama/31688).
  • Intel / AMD / Qualcomm (Vulkan / SYCL / OpenCL / Hexagon): Vulkan FlashAttention prefetch kernels were added for Intel GPUs (llama/29357), and fused SDPA (head_dim=64) restrictions for Alchemist were lifted in SYCL environments (llama/31574). For AMD, large-tile split-K optimizations for RDNA3/4 and large coopmat matmul on recent AMD proprietary drivers (26.10.1+) were enabled (llama/30280, llama/31587). For Qualcomm, decoupled FlashAttention was added for Adreno (llama/32046), alongside conv_2d optimizations and conv_3d support on Hexagon DSPs (llama/31814).

Model Architecture and Processor Support

In terms of model architecture, the update handles fused MoE layer processing for DeepSeek-V4/V4.1 and GLM, and adds kernel routing for GATED_DELTA_NET targeting SpacemiT (RVV) processors (llama/31897). Both operational functionality and performance have been enhanced in edge and ARM environments through utilization of 4×4 simd_gemm tiles in ARM NEON (llama/32128) and host build support via clang-cl on Windows ARM64 (llama/32011).

How to Get It

To build and install ggml v1.0.0 from source, run the following steps:

git clone https://github.com/ggml-org/ggml
cd ggml

mkdir build && cd build
cmake..
cmake --build. --config Release -j 8

If you have an existing build environment, fetch the latest repository, check out the v1.0.0 tag, and re-compile.

Related Articles

What to Read Next

Sources