Magnitude CLI 0.2.7 Released: Major Speedups and Fixes

At a Glance
| Item | Value |
|---|---|
| Repository | magnitudedev/magnitude |
| Version | @magnitudedev/cli@0.2.7 |
| Published | 2026-10-12 |
| License | Apache-2.0 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
@magnitudedev/cli@0.2.7, a CLI tool and desktop environment update for the agent-oriented inference engine magnitude, has been released. magnitude is a Rust-based inference engine (licensed under Apache-2.0) that automatically compiles and tunes kernels to match the hardware of the execution environment, enabling fast local execution of open models.
This version brings significant inference speedups on Apple Silicon and NVIDIA GPU environments. Decoding speed when processing multiple concurrent requests has improved by up to approximately 3x, and prompt processing speed by up to 4.5x, making this a release with substantial benefits for users who run local models on a daily basis.
Key Changes
Major Local Inference Speedups and Enhanced Speculative Decoding
Overall inference performance has been significantly boosted for Apple Silicon and NVIDIA GPU targets. In addition to a decoding speedup of up to ~3x during concurrent execution of multiple requests, prompt processing speeds up to 4.5x have been confirmed with Gemma 4 E2B on Apple Silicon. Furthermore, a configuration utilizing assistant models as draft heads has been introduced for Gemma 4 models, and speculative decoding with DFlash drafters has been enabled for Qwen3.5 4B and 9B models, substantially improving generation speeds. For Apple Silicon Macs ranging from M1 to M4, optimizations such as a new tiled weight layout, token packing, and simultaneous utilization of matrix and scalar units have been incorporated to improve both prefill and decode. The optimization start time upon initial model loading has also been reduced; for example, Qwen 3.8 27B improved from 12.6 seconds to 1.9 seconds.
Arch Linux Support and Improved User Experience on Non-Supported GPU Environments
Pacman packages are now provided for Arch Linux, Omarchy, and other Arch-based distributions (x64). This allows smooth local deployment and in-app updates via pacman. Consideration for PCs without supported GPUs (such as x86 environments with Intel processors) has also been enhanced. Text generation responses are now streamed incrementally rather than returned all at once after completion, and request cancellations or aborts are reflected more promptly. These improvements boost waiting times and UI responsiveness even on environments lacking dedicated GPUs.
Long-Running Memory Management Improvements and GPU Crash Fixes
Multiple stability bugs across various GPU environments have been resolved. On Mac environments, an issue where models were prematurely unloaded due to out-of-GPU-memory errors during long-running agent executions has been fixed by releasing memory from previous turns in advance. On NVIDIA GPU environments, crashes caused by illegal memory accesses when processing multiple concurrent requests after long prompts, loading failures for Qwen 3.8 27B, and memory errors with Gemma 4 E2B/E4B have been resolved. For AMD GPUs on Linux, driver timeout limits caused by massive images or long prompts leading to GPU resets have been addressed by splitting operations to ensure stable execution. Additionally, an issue on Windows where initial optimization processes failed to run, leaving kernels unoptimized and slow, has been fixed.
Agent Integration, Tool Calling Bug Fixes, and Behavior Improvements
Operations during agent features and various tool-calling scenarios have been stabilized. In OpenClaw integration, an issue where the automations tool monopolized the server and caused the model to be evicted from memory and hang has been resolved, allowing responses to be returned within seconds. Also fixed were issues where requests from Claude Code or Oh My Pi using MiniCPM5 or LFM2.5 encountered “Too many items" errors, as well as failures when disabling reasoning in Qwen3.5. If output token limits are reached mid-tool-call, the process is now safely completed under normal length limits (length, max_tokens, incomplete) rather than exiting with an error. Furthermore, an issue where retries always returned the same result when no seed was specified in a request has been fixed, changing it to sample with a new seed value every time.
Supported Models and Hardware
This version includes optimizations and bug fixes for multiple model architectures, alongside improved support for various GPU and CPU environments.
Model-Specific Support and Behavior Improvements
- Gemma 4 Series: Enabled fast generation utilizing assistant models as draft heads. Resolved an issue where empty answers were returned when reasoning was turned off via the Responses API, and fixed illegal memory accesses that occurred when running Gemma 4 E2B and E4B on NVIDIA GPUs. Additionally, an issue where Gemma 4 12B remained at slow default speeds (4.8 tok/s) when run with a long context of 65,536 tokens has been fixed, enabling reliable optimizations.
- Qwen Series: Qwen3.5 4B and 9B models now support speculative decoding using DFlash drafters, improving generation speed. An issue on Mac environments where initial optimizations failed to complete and re-ran on every model load has been resolved. Furthermore, Qwen 3.8 27B can now be loaded normally on NVIDIA GPUs, with reduced optimization start times on initial load.
- MiniCPM5 and LFM2.5: Fixed an issue causing requests from Claude Code or Oh My Pi to fail with “Too many items" errors, allowing normal dialogue.
- Muse Glimmer: Toggling reasoning on and off is now supported.
- All Vision Models: Input image processing limits have been revised. Large images that previously failed when exceeding engine limits can now be encoded up to 4,096 cells, and images larger than that are automatically resized to fit within bounds.
Hardware and Platform Support
- Apple Silicon Macs (M1–M4): Both prefill and decode have been accelerated via a new tiled weight layout, token packing during prompt processing, and cooperative operation between matrix and scalar units. Furthermore, macOS can now load models immediately after download without rejection, provided they fit within available wired or compressed memory.
- NVIDIA GPU: Crashes due to illegal memory accesses during parallel processing of multiple requests after long prompts have been fixed, enabling stable batch processing.
- AMD GPUs on Linux: Resolved issues where driver timeout limits were exceeded during large image or long prompt processing, causing GPU resets and unloads. Operations are now appropriately split, and out-of-memory failures for image requests after long sessions have been resolved. Initial loading now prepares all kernels, speeding up first-time usage.
- Windows Environment: Fixed an issue where initial optimizations were skipped, resulting in un-tuned slow kernels. Issues involving disk space not being freed upon model deletion and Codex connection problems have also been resolved.
- Environments Lacking Supported GPUs (Intel and Other x86 CPUs): Generation processing itself has been accelerated on x86 processors such as Intel, alongside support for streaming output. Issues where optimization failed on 8GB memory GPUs or unsupported GPU environments have also been fixed.
- Arch Linux (x64): Official pacman packages are now provided for Arch Linux, Omarchy, and other Arch-based distributions.
How to Get It
If you are using Arch Linux or Arch-based distributions (x64), obtain the distributed package file (magnitude-desktop.pkg.tar.zst) and run the following pacman command to install or update:
sudo pacman -U./magnitude-desktop.pkg.tar.zst
Installation using the official installation script is also available. In-app updates after deployment are handled via pacman. Note that similar to Debian and RPM packages, pacman upgrades or removals are rejected while the magnitude process is running, so please completely terminate the application before performing updates.
If you are running in the background on existing environments (e.g., using nohup magnitude serve &), it is recommended to terminate the server process once, apply the latest version, and restart.
For binaries targeting other platforms and detailed installation instructions, please check the release page.
Related Articles
- Magnitude CLI 0.2.8 Released with VRAM Fixes and Gemma 4 Support
- Magnitude CLI 0.2.14 Released: Faster CPU Decoding and More
- Magnitude @magnitudedev/cli@0.2.12 Released
- Magnitude CLI 0.2.11 Released with Custom GGUF Support
What to Read Next
- Follow this tool → Magnitude overview and release history (16 releases tracked)
- Other inference engines and runtimes → llama.cpp / Ollama / vLLM

