{"id":11866,"date":"2026-11-17T11:00:39","date_gmt":"2026-11-17T02:00:39","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=11866"},"modified":"2026-11-17T11:09:50","modified_gmt":"2026-11-17T02:09:50","slug":"ggml-v0-27-0-released-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ggml-v0-27-0-released-2\/","title":{"rendered":"ggml v0.27.0 Released with Tensor-Parallel RPC and Major Speedups"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\">ggml-org\/ggml<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.27.0\">v0.27.0<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-15<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>MIT<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>Version 0.27.0 of ggml, the C++ tensor library for machine learning, has been released. ggml serves as the low-level core library for local LLM execution runtimes such as llama.cpp.<\/p>\n<p>This release updates the RPC protocol to major version 8, introduces a tensor-parallel RPC server mode, and brings substantial performance-boosting kernels across major backends including CUDA, CPU, Vulkan, SYCL, and Metal. This is an important update for users performing distributed inference over a network or those looking to squeeze maximum inference speed out of the latest quantization methods and architectures on various hardware.<\/p>\n<h2>Breaking Changes and Deprecations<\/h2>\n<p>The following changes have been made regarding the RPC communication protocol and library symbol visibility.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Old (v0.26.0 and earlier)<\/th>\n<th>New (v0.27.0)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>RPC Protocol<\/td>\n<td><code>RPC_PROTO_MAJOR_VERSION<\/code> 7<\/td>\n<td><code>RPC_PROTO_MAJOR_VERSION<\/code> 8<\/td>\n<\/tr>\n<tr>\n<td>Default Symbol Visibility<\/td>\n<td>Public (exports unintended internal symbols as well)<\/td>\n<td>Hidden (unnecessary internal symbols are no longer exported)<\/td>\n<\/tr>\n<tr>\n<td>Duplicate expert specification for <code>ggml_mul_mat_id<\/code><\/td>\n<td>Undefined (implementation-dependent looping behavior, etc.)<\/td>\n<td>Explicitly specified as &#8220;undefined&#8221; in the spec<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>In environments performing distributed execution across multiple machines using RPC features, both clients and servers must be updated to v0.27.0 simultaneously. Because the protocol major version has been incremented, mixed communication with older versions will not work. Additionally, developers directly linking ggml as a shared library should verify that they do not depend on internal symbols that have now been hidden.<\/p>\n<h2>Key Changes<\/h2>\n<h3>RPC Tensor-Parallel Server Support<\/h3>\n<p>A tensor-parallel server mode has been introduced to the RPC (Remote Procedure Call) backend. It supports asynchronous graph computations, custom all-reduce operations, and 2D tensor transfers. This improves efficiency in environments where tensors are distributed across multiple compute nodes via a network to cooperatively run large-scale models. Furthermore, the environment variable <code>GGML_RPC_DEBUG<\/code> is now treated as a verbosity level, improving the readability of transport logs and tensor transfer logs.<\/p>\n<h3>Scheduler API Extensions<\/h3>\n<p>Two callback functions have been added to the backend scheduler: a copy callback (<code>ggml_backend_sched_set_copy_callback<\/code>) allowing users to control split weight copies on the application side, and a refactoring callback (<code>ggml_backend_sched_set_realloc_callback<\/code>) to notify about compute buffer reallocations during inference. This enables applications to exercise fine-grained control over multi-device memory layouts and dynamic buffer management.<\/p>\n<h3>Expansion of Optimization Paths in Various Backends<\/h3>\n<p>Extensive performance-boosting kernels have been added for various hardware accelerators.<br \/>\n&#8211; <strong>CUDA \/ HIP<\/strong>: Chunked kernels for gated-delta-net were introduced, increasing the number of state columns per warp. Support for 64-row MMQ tiles for RDNA3.5, BF16 support for XIELU, and improvements to top-k algorithm selection were also implemented.<br \/>\n&#8211; <strong>CPU<\/strong>: An AVX2 dot product kernel for Q2_0 was added. Furthermore, matrix multiplication for Q4_K is now supported in Arm SME2 environments via KleidiAI, and Q6_K is optimized to route through the Q8_0 path. FP16-to-FP32 lookup tables when F16C is enabled on x86 environments have also been restored.<br \/>\n&#8211; <strong>Vulkan<\/strong>: L2 cache-aware workgroup scheduling was implemented for coopmat2 matrix multiplication, along with support for bf16 activations in mul_mat. On AMD environments, an optimization was added to copy strided FP16 KV caches once and distribute reads across memory channels.<br \/>\n&#8211; <strong>SYCL<\/strong>: Added weight reordering and arithmetic decoding for MXFP4, IQ3, and Q5_K to accelerate MoE, grouped MoE XMX GEMM, and faster MLA prefill using MKL flash attention.<br \/>\n&#8211; <strong>Metal<\/strong>: Tensor API flash attention kernels for BF16 were added, making head-dimension 128\/96 kernels available across data types.<br \/>\n&#8211; <strong>OpenCL<\/strong>: dp4a-based binary GEMM kernels for MoE were introduced, alongside expanded head dimension support for flash attention.<\/p>\n<h3>Correctness and Memory Safety Fixes<\/h3>\n<p>A bug in <code>ggml_acc<\/code> and <code>ggml_set<\/code> where negative offset values bypassed boundary checks during sign extension, causing out-of-bounds writes, has been fixed. Additionally, the output layout of <code>ggml_conv_1d<\/code> during batch inputs was fixed to correctly be <code>[N, OC, OL]<\/code>, and CLAMP operations on non-contiguous views were revised to work properly in the CPU and CUDA backends.<\/p>\n<h2>Supported Models and Hardware<\/h2>\n<p>ggml v0.27.0 brings optimizations and feature additions for numerous processor architectures and quantization formats. The specific support details that users can benefit from on their hardware environments are as follows:<\/p>\n<h3>Expansion of Quantization Formats<\/h3>\n<ul>\n<li><strong>MXFP4 \/ IQ3 \/ Q5_K<\/strong>: Weight reordering and arithmetic decoding for Mixture of Experts (MoE) calculations were introduced in the SYCL backend.<\/li>\n<li><strong>Q2_0<\/strong>: An AVX2 dot product kernel was added to the CPU backend, improving processing efficiency for low-bit quantized models.<\/li>\n<li><strong>Q4_K \/ Q6_K<\/strong>: Support for Q4_K matrix multiplication in Arm SME2 environments (via KleidiAI) and dp4a-based Q4_K \/ Q6_K (vs Q8_1) MoE binary GEMM kernels in OpenCL environments were added. GET_ROWS tiling processing for Q4_K \/ Q6_K was also implemented for Hexagon DSP.<\/li>\n<li><strong>Q8_0<\/strong>: Support for Q8_0 matrix multiplication was newly added in the zDNN backend (for IBM zSystems).<\/li>\n<\/ul>\n<h3>Enhanced Support for Model Architectures<\/h3>\n<ul>\n<li><strong>MoE (Mixture of Experts) Models<\/strong>: MoE inference processing has been enhanced across backends, including grouped XMX GEMM in SYCL and the addition of dp4a kernels in OpenCL (such as Qualcomm Adreno).<\/li>\n<li><strong>MLA (Multi-head Latent Attention) Models<\/strong>: Faster MLA prefill using MKL flash attention was added to the SYCL backend, streamlining context processing for models adopting the MLA architecture.<\/li>\n<li><strong>Gated-Delta-Net Models<\/strong>: Chunked kernels were introduced for CUDA and HIP environments, and arithmetic fusion of alpha gates (add + unary + mul) was performed in SYCL.<\/li>\n<li><strong>Flash Attention Extensions<\/strong>: The supported range of head dimensions was expanded in the OpenCL backend to handle configurations such as dk512. Head dimension 128 and 96 kernels were also established in the Metal backend, along with the addition of BF16 Tensor API kernels.<\/li>\n<\/ul>\n<h3>Hardware Accelerator Support<\/h3>\n<ul>\n<li><strong>CPU (x86 \/ Arm)<\/strong>: KleidiAI acceleration for Arm SME2, and F16C detection fixes and FP16-to-FP32 lookup table restoration for AVX2 environments on x86.<\/li>\n<li><strong>CUDA \/ AMD HIP<\/strong>: 64-row MMQ tiles are supported for select quantization types on the AMD RDNA3.5 architecture. Adjustments were made to prioritize MMVQ when executing MUL_MAT_ID on older NVIDIA architectures (sm_60).<\/li>\n<li><strong>Vulkan<\/strong>: Shader counts were defined for Intel Arc Pro B65 and B70, and fixes were applied to address memory read imbalances and slowdowns on AMD GPUs and integrated GPUs (iGPUs). Sparse flash attention via coopmat2 and L2 cache-optimized scheduling are also available.<\/li>\n<li><strong>Qualcomm Hexagon DSP<\/strong>: A feature to configure HTP DCVS\/HMX power modes via the <code>GGML_HEXAGON_POWER_MODE<\/code> environment variable was added. Major revisions to copy processing utilizing DMA and HVX, as well as the addition of IM2COL_3D and pooling processing, were performed.<\/li>\n<li><strong>Others<\/strong>: WebGPU backend now supports 2D workgroup dispatch, and OpenVINO backend received fixes for GPU stateful execution.<\/li>\n<\/ul>\n<h2>How to Get It<\/h2>\n<p>ggml is a C++ library built from source. To build and use the latest version in your local environment, run the following steps:<\/p>\n<pre><code class=\"language-bash\">git clone https:\/\/github.com\/ggml-org\/ggml\ncd ggml\n\nmkdir build &amp;&amp; cd build\ncmake..\ncmake --build. --config Release -j 8\n<\/code><\/pre>\n<p>If you are using an existing repository, run <code>git fetch<\/code> and <code>git checkout v0.27.0<\/code> inside your working directory before rebuilding.<\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ggml-v1-0-0-released\/\">ggml v1.0.0 Released: Major Version Update and Improvements<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/06\/ggml-v0260-released\/\">ggml v0.26.0 Released with Sparse Flash Attention &amp; New Backends<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/ggml-v0290-released\/\">GGML v0.29.0 Released: Multi-Node Tensor Parallelism and More<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/24\/ggml-v0-25-2-released\/\">ggml v0.25.2 Released with Hardware Backend Optimizations<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Follow this tool<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ggml-en\/\">ggml overview and release history (45 releases tracked)<\/a><\/li>\n<li><strong>Other quantization, model formats and fine-tuning<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-exllamav3-en\/\">ExLlamaV3<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-unsloth-en\/\">Unsloth<\/a><\/li>\n<li><strong>Engines mentioned in this article<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.27.0\">ggml-org\/ggml v0.27.0 Release Notes<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/26610\">llama.cpp PR #26610 (RPC tensor-parallel server mode)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/29943\">llama.cpp PR #29943 (Backend sched copy callback)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30329\">llama.cpp PR #30329 (Backend sched realloc callback)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30014\">llama.cpp PR #30014 (Change default symbol visibility to hidden)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30501\">llama.cpp PR #30501 (Document mul_mat_id undefined duplicate behavior)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/29353\">llama.cpp PR #29353 (Chunked gated-delta-net kernels)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/29219\">llama.cpp PR #29219 (KleidiAI SME2 Q4_K and Q6_K matmul)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30447\">llama.cpp PR #30447 (AVX2 dot product for Q2_0)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30491\">llama.cpp PR #30491 (Metal BF16 tensor API flash attention)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30215\">llama.cpp PR #30215 (Vulkan L2-friendly workgroup scheduling)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/29809\">llama.cpp PR #29809 (SYCL MXFP4 MoE acceleration)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30187\">llama.cpp PR #30187 (OpenCL dp4a MoE binary GEMM kernels)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/30423\">llama.cpp PR #30423 (Hexagon HTP DCVS\/HMX power modes)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/29836\">llama.cpp PR #29836 (zDNN Q8_0 matmul support)<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>ggml v0.27.0 is out, featuring major RPC updates, tensor-parallel server mode, and extensive performance boosts across CUDA, CPU, Vulkan, SYCL, and Metal.<\/p>\n","protected":false},"author":1,"featured_media":11865,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[505,1187,161,163,1073,1547,511],"class_list":["post-11866","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-apple-silicon-en","tag-cuda-en","tag-ggml-en","tag-gguf-en","tag-llama-cpp-en","tag-verified","tag-vulkan-en"],"lang":"en","translations":{"en":11866,"ja":11864},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11866","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=11866"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11866\/revisions"}],"predecessor-version":[{"id":17803,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11866\/revisions\/17803"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/11865"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=11866"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=11866"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=11866"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}