{"id":10268,"date":"2026-10-06T07:14:59","date_gmt":"2026-10-05T22:14:59","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/10\/06\/magnitudedev-magnitude-026-released\/"},"modified":"2026-10-06T07:14:59","modified_gmt":"2026-10-05T22:14:59","slug":"magnitudedev-magnitude-026-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/06\/magnitudedev-magnitude-026-released\/","title":{"rendered":"magnitudedev\/magnitude v0.2.6 Released: Major Mac Speedups"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nmagnitudedev\/magnitude<\/a><\/td>\n<\/tr>\n
Version<\/td>\n@magnitudedev\/cli@0.2.6<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-10-06<\/td>\n<\/tr>\n
License<\/td>\nApache-2.0<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code when the information was collected. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

The latest version of magnitudedev\/magnitude<\/code>, @magnitudedev\/cli@0.2.6<\/code>, has been released. magnitudedev\/magnitude<\/code> is an open-source inference engine written in Rust (licensed under Apache-2.0) that automatically compiles, tunes, and optimizes kernels to match the execution hardware. It supports Apple Silicon, NVIDIA, AMD, and CPU, and features the ability to run open models up to 2x faster compared to llama.cpp.<\/p>\n

The most significant changes in this update are a dramatic improvement in prompt processing speed on Mac environments (especially M5 and later chips, and Apple Silicon generations such as M1\/M2\/M4 Pro) and enhanced stability during long-prompt processing. For users running local LLMs on Macs, this is a highly valuable update in terms of both performance and stability.<\/p>\n

Key Changes<\/h2>\n

Dramatically Faster Prompt Processing on Mac Environments<\/h3>\n

Prompt processing speed has improved by approximately 50% on M5 and later Macs. For example, when running Qwen3.5-4B with a 64K context length, prompt processing speed has increased from the previous 652 tok\/s to approximately 970 tok\/s. This is due to prompt attention processing directly reading key-value tensors through GPU tensor operations without staging, as well as improved kernel tuning that avoids maintaining unstable and slow default settings.<\/p>\n

Additionally, for all Macs, weight block decoding has been changed from every 64 rows to once every up to 512 rows. Without altering the output results, this speeds up matrix multiplication by 7\u201311% on M4 Pro and 17\u201325% on M1. On M4 Pro with Qwen3.5-4B (64K context), prompt processing improves from the previous 513 tok\/s to 534 tok\/s.<\/p>\n

Fixed Crashes During Long-Context Processing on M1\/M2 Macs<\/h3>\n

A bug has been fixed where M1 and M2 Macs encountered a “device lost: \u2026 Impacting Interactivity” error while processing long prompts, leaving the model unloaded.<\/p>\n

In this version, similar to llama.cpp, processing has been added to relax macOS GPU watchdog limits at startup. As a result, when using Gemma 4 26B, long prompt processing of 35k tokens and 69k tokens\u2014which previously failed around the 20k token mark\u2014can now complete without failing midway.<\/p>\n

Resolved Metal Compilation Errors on Certain Macs<\/h3>\n

An issue where loading models such as Qwen 3.6 on certain Mac environments like M5 Max failed with a Metal shader compilation error reading “no template named ‘extents’ in namespace ‘metal'” has been fixed. Through the fix in PR #166<\/a>, compiling Metal kernels now uses the exact same language version confirmed during device detection, allowing kernels utilizing tensor operations to build successfully on supported devices.<\/p>\n

Prevention of 502 Timeouts on Long-Running Requests<\/h3>\n

An issue where long-running processing requests for local models failed with a 502 error after approximately 5 minutes has been fixed. For non-streaming generation and long prompt processing, data is not transmitted until generation is complete, but improvements were made so that connections to the engine do not time out while waiting.<\/p>\n

Prevention of Force Quits Due to JSON Errors on Windows<\/h3>\n

An issue has been fixed where the engine abnormally aborted on Windows environments when chat templates or tool calls generated invalid JSON. Enabling C++ exception handling during the MSVC build allows the engine to properly report JSON errors without crashing.<\/p>\n

Various Bug Fixes for Tool Calling and Session Processing<\/h3>\n

Multiple bugs related to agent features and API integrations have been fixed.<\/p>\n