Magnitude CLI 0.2.10 Released with Memory Fixes and New Quantizations

At a Glance
| Item | Value |
|---|---|
| Repository | magnitudedev/magnitude |
| Version | @magnitudedev/cli@0.2.10 |
| Published | 2026-10-20 |
| License | Apache-2.0 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
Version 0.2.10 of Magnitude CLI, the command-line tool for the inference engine “Magnitude", has been released. Magnitude is an open-source inference engine written in Rust (licensed under Apache-2.0) that tunes kernels on the hardware of the execution environment to apply methods for running each model at maximum speed. It operates on Apple Silicon, NVIDIA, AMD, and CPU, and is characterized by being twice as fast as Ollama.
This version includes fixes for memory calculation and management processing in unified memory, integrated GPUs, and Linux and virtual machine environments. Bugs related to model loading failures and memory overcommitment have been improved, alongside enhanced support for loading models with specific quantization formats and output heads.
Key Changes
Fixes to Memory Planning in Unified Memory and Integrated GPUs
In unified memory environments such as NVIDIA GB10 and Apple Silicon, as well as integrated GPU environments, memory calculation for one-time optimization processing has been improved. It has been fixed to count the memory already used by the loaded model without over-allocating (overcommitting).
In integrated GPU environments like AMD Strix Halo, improvements have been made to continue processing by utilizing shared memory rather than stopping when the dedicated carve-out area becomes full. Additionally, a mechanism has been added to retry memory allocation rather than failing immediately if the GPU rejects it. Furthermore, each GPU is now listed once without duplication on the app, displaying the backend being used by Magnitude.
Stabilization of Model Loading in Linux Environments and Virtual Machines
Issues causing model loading failures on physical Linux and virtual machine (VM) environments have been fixed. Because it has been changed to accurately count the memory held by OS file caches as available area, instances where processing fails due to insufficient memory despite the system having releasable memory are avoided.
If a true out-of-memory condition occurs at startup, it is improved to be notified as a retryable issue rather than failing midway through loading. Furthermore, in CPU environments lacking the instruction sets required by the inference engine, a clear warning message is output before starting to load the model.
Bug Fixes for Quantization Formats and Specific Model Output Heads
Models with IQ2_XXS weights are now supported across all backends. At the same time, an issue where loading models with IQ2_XS weights failed on Mac environments has been resolved.
Additionally, loading failures for models with Q8_0 output heads have been fixed. This includes MiniCPM5 1B Q8, as well as Qwen3.6 35B-A3B and Qwen3.5 9B Q6 with drafting functions turned off. Furthermore, fast output settings for models with large vocabularies have become selectable once again in one-time optimization processing.
Web App UI Display Improvements and RPM Package Conflict Avoidance
In the web application interface, an explanation stating “Models are automatically loaded upon the first request" has been added to the model status display. Additionally, the Update button now includes guidance explaining what changes the update will bring.
For Linux environments, a bug where the RPM package conflicted with other installed applications over a shared build ID has been fixed.
Supported Models and Hardware
This version aims to maximize the characteristics of the latest hardware and improve compatibility with models having specific quantization formats. The available models and hardware environments verified to work are as follows:
Supported Models and Quantization Formats
- IQ2_XXS Weight Models: Models with the extremely high compression rate IQ2_XXS quantization format are now supported across all calculation backends. This expands options in environments with very limited resources.
- IQ2_XS Weight Models (Mac): The issue where models with IQ2_XS weights previously failed to load in Mac environments has been fixed, allowing them to operate normally on devices such as Apple Silicon.
- Models Employing Q8_0 Output Heads: Support has been added for loading models where Q8_0 quantization is applied to the output head, which corresponds to the final layer of the model. Specifically, this supports the operation of MiniCPM5 1B Q8, as well as Qwen3.6 35B-A3B and Qwen3.5 9B Q6 with the drafting (inference acceleration) feature disabled.
- Large Vocabulary Models: For models with a large vocabulary size, faster output methods can now be selected once again during the one-time optimization process.
Optimized and Improved Hardware Environments
- NVIDIA GB10 and Apple Silicon: In unified memory environments, memory planning has been improved to accurately count the amount of memory already used by the model, avoiding excessive allocation (overcommitment) exceeding physical memory.
- AMD Strix Halo (Integrated GPU): Even if the dedicated memory area (carve-out) specific to integrated GPUs becomes full, processing no longer stops; instead, system shared memory can be used in tandem to continue inference. This enables running models of sizes that exceed integrated GPU limitations.
- Linux and Virtual Machines (VM): Fixed to correctly recognize OS file cache areas as available memory. This resolves instances where loading fails due to insufficient memory despite the system having releasable memory.
- CPU Instruction Set Requirements: If the execution environment’s CPU lacks the instruction sets required for Magnitude to operate, users are now notified with a clear error message before model loading begins. This prevents unexpected crashes in incompatible environments.
- GPU Backend Visualization: Each GPU device is listed once on the application, explicitly indicating which backend (calculation foundation) Magnitude assigns and uses for each device.
How to Get It
Update procedures for Magnitude CLI vary depending on your OS and package format.
For Linux (RPM Package) Users
In this version, an issue where shared build IDs conflicted with other installed applications causing malfunctions has been fixed. Users who experienced conflicts in previous versions should reinstall the latest RPM package.
Other Environments and Web App Version
Specific CLI update commands using package managers like pip or npm are not specified in the documentation. For the latest binaries, source code, and packages for each platform, please obtain them from the GitHub releases page.
If you are using the web app version, you can click the “Update" button on the interface to review explanations of changes brought by the update and apply the update. Note that models are specified to be automatically loaded upon the first request.
Related Articles
- Magnitude CLI 0.2.14 Released: Faster CPU Decoding and More
- Magnitude CLI 0.2.11 Released with Custom GGUF Support
- magnitudedev/magnitude v0.2.6 Released: Major Mac Speedups
- Magnitude CLI v0.2.5 Released: M5 Mac & MoE Speedups
What to Read Next
- Follow this tool → Magnitude overview and release history (16 releases tracked)
- Other inference engines and runtimes → llama.cpp / Ollama / vLLM
Sources
- https://github.com/magnitudedev/magnitude/releases/tag/%40magnitudedev/cli%400.2.10
- https://github.com/magnitudedev/magnitude/issues/195
- https://github.com/magnitudedev/magnitude/issues/196
- https://github.com/magnitudedev/magnitude/issues/199
- https://github.com/magnitudedev/magnitude/issues/200
- https://github.com/magnitudedev/magnitude/issues/206
- https://github.com/magnitudedev/magnitude/issues/207

