{"id":631,"date":"2026-09-14T23:17:56","date_gmt":"2026-09-14T14:17:56","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/14\/ggml-v0-24-0-released-2\/"},"modified":"2026-09-18T21:42:02","modified_gmt":"2026-09-18T12:42:02","slug":"ggml-v0-24-0-released-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/14\/ggml-v0-24-0-released-2\/","title":{"rendered":"ggml v0.24.0 Released with Backend Improvements and API Updates"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\">ggml-org\/ggml<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.24.0\">v0.24.0<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-14<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>MIT<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code at collection time. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>ggml-org\/ggml v0.24.0 has been released. ggml is a machine learning tensor library (written in C++, MIT license, 15,352 stars) used as the foundation for many local LLM inference engines.<\/p>\n<p>This version focuses on expanding backend coverage and robustness, introducing a new precision control API, major backend improvements for Vulkan, SYCL, Hexagon, and OpenCL, alongside numerous fixes and performance enhancements across various backends.<\/p>\n<h2>Key Changes<\/h2>\n<ul>\n<li>Expanded the new precision control API <code>ggml_prec<\/code> with <code>GGML_PREC_BF16<\/code>, <code>F16<\/code>, <code>Q8<\/code>, and <code>Q4<\/code> (<code>GGML_PREC_DEFAULT<\/code> is now deprecated)<\/li>\n<li>Added <code>ggml_prec_set_acc()<\/code> and <code>ggml_prec_set_src()<\/code> to control the precision of <code>MUL_MAT<\/code>, <code>MUL_MAT_ID<\/code>, and flash attention, deprecating the older <code>ggml_mul_mat_set_prec()<\/code> and <code>ggml_flash_attn_ext_set_prec()<\/code><\/li>\n<li>Backend scheduler changes prevent unnecessary splitting of backend inputs and skip size-0 MoE <code>ids<\/code> tensors<\/li>\n<li>Added PCH (Precompiled Header) and unity-build support to reduce build times<\/li>\n<\/ul>\n<h2>Supported Models and Hardware<\/h2>\n<ul>\n<li><strong>CPU<\/strong>: Added repacked support for <code>q4_0<\/code> and Q1_0 vector intrinsics on s390x, guarded VXE-specific helpers, and added non-VXE tests<\/li>\n<li><strong>CUDA \/ HIP \/ MUSA<\/strong>: Added dedicated MMQ configuration tables for AMD GCN and gfx90c HIP support. Devices without hardware acceleration for BF16 now fall back to F32, and flash attention tuning was added for gfx1201<\/li>\n<li><strong>Metal<\/strong>: Added vector tuning for flash attention on M2 Max and M3<\/li>\n<li><strong>Vulkan<\/strong>: Added support for TQ1_0 and <code>iq4_xs<\/code>, type-aligned <code>GET_ROWS<\/code>, f16 B-type matrix multiplication pipelines, and DeepSeek-V4 hyper-connection fused operations<\/li>\n<li><strong>OpenCL<\/strong>: Added Adreno xmem SDPA path and A8 <code>q4_0<\/code>\/<code>q4_K<\/code> binary GEMM kernels<\/li>\n<li><strong>SYCL<\/strong>: Added radix-select for <code>top_k<\/code>, batched <code>L2_NORM<\/code>, and IQ type handling for MoE<\/li>\n<li><strong>Hexagon<\/strong>: Added multi-device model splitting (row-split) support, along with <code>RELU<\/code> and <code>LEAKY_RELU<\/code> operations<\/li>\n<li><strong>WebGPU<\/strong>: Added Dawn updates and type-aligned <code>GET_ROWS<\/code> support<\/li>\n<\/ul>\n<h2>How to Get It<\/h2>\n<p>To build from source, run the following commands:<\/p>\n<pre><code class=\"language-bash\">git clone https:\/\/github.com\/ggml-org\/ggml\ncd ggml\n\nmkdir build &amp;&amp; cd build\ncmake..\ncmake --build. --config Release -j 8\n<\/code><\/pre>\n<p><!-- lmw:releases --><\/p>\n<h2>Releases Since Our Last Article<\/h2>\n<p><em>Compiled by Local Model Watch from the project&#8217;s GitHub releases: the versions between this release and the last one we covered, which did not get separate articles.<\/em> <em>Full history: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ggml-en\/\">release tracker<\/a>.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Version<\/th>\n<th>Released<\/th>\n<th>Release notes<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>v0.23.0<\/td>\n<td>2026-09-04<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.23.0\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v0.22.0<\/td>\n<td>2026-08-25<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.22.0\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v0.21.0<\/td>\n<td>2026-08-21<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.21.0\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v0.20.2<\/td>\n<td>2026-08-18<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.20.2\">GitHub<\/a><\/td>\n<\/tr>\n<tr>\n<td>v0.20.1<\/td>\n<td>2026-08-17<\/td>\n<td><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.20.1\">GitHub<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:releases --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.24.0\"><a href=\"https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.24.0\">https:\/\/github.com\/ggml-org\/ggml\/releases\/tag\/v0.24.0<\/a><\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>ggml v0.24.0 is out, featuring a new precision control API, major backend improvements for Vulkan, SYCL, and Metal, and faster build times.<\/p>\n","protected":false},"author":1,"featured_media":630,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[1187,161,163,136,1189,511,117],"class_list":["post-631","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-cuda-en","tag-ggml-en","tag-gguf-en","tag-llm-en","tag-metal-en","tag-vulkan-en","tag--en"],"lang":"en","translations":{"en":631,"ja":629},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/631","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=631"}],"version-history":[{"count":7,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/631\/revisions"}],"predecessor-version":[{"id":1753,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/631\/revisions\/1753"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/630"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=631"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=631"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=631"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}