{"id":15605,"date":"2026-11-17T11:03:52","date_gmt":"2026-11-17T02:03:52","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=15605"},"modified":"2026-11-17T11:13:24","modified_gmt":"2026-11-17T02:13:24","slug":"magnitude-cli-0-2-17-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-17-released\/","title":{"rendered":"Magnitude CLI v0.2.17 Released with Dynamic Context Scaling"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\">magnitudedev\/magnitude<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/@magnitudedev\/cli@0.2.17\">@magnitudedev\/cli@0.2.17<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-11-09<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>Apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>Version 0.2.17 of the CLI tool <code>@magnitudedev\/cli<\/code> for the open-source inference engine Magnitude has been released. Magnitude is an Apache-2.0 licensed inference engine developed in Rust, designed to tune kernels according to hardware and optimize models to run at maximum speed.<\/p>\n<p>The most significant change in this version is the ability to launch models by automatically adjusting the maximum available context length according to the machine&#8217;s available memory. This ensures that models which would otherwise run out of memory at full context can still be used safely within sizes fitting 32K or more, without being rejected at startup. Additionally, inference processing has been significantly accelerated in ARM CPU environments, and the accuracy of GPU memory estimation has been improved.<\/p>\n<h2>Key Changes<\/h2>\n<h3>Dynamic Context Length Adjustment Based on Memory Capacity<\/h3>\n<p>For models where the full context does not fit into the computer&#8217;s memory, the previous startup rejection has been changed to a method where the model launches by providing the maximum context length that fits within a range from 32K up to the full length. The Discover feature prioritizes and recommends the highest quality version reaching 64K context, falling back to 32K as needed.<\/p>\n<p>This change affects chat functionality, the <code>\/v1\/models<\/code> API, and various connected agents (Claude Code, Codex, OpenCode). For example, Claude Code does not uniformly assume a 200K context, but automatically compresses before reaching the provided context length. Prompts exceeding the provided context length are rejected with a <code>context_length_exceeded<\/code> error. Since the memory check is performed before loading the model, there is no need to worry about the model being forcibly loaded at an unsupported size.<\/p>\n<h3>Model Memory Breakdown Display (Weights, Context, Overhead)<\/h3>\n<p>As a breakdown of the memory consumed by a model, you can now individually check weights, context, and overhead. Within the app, the breakdown can be checked from the information icon next to the memory display on the model card, or from next to Fit on the Discover radar.<\/p>\n<p>The breakdown is also included in the CLI commands <code>magnitude catalog show<\/code> and <code>magnitude models status<\/code>, as well as their <code>--json<\/code> output. The <code>magnitude hardware<\/code> command also displays each memory item of the loading model individually using the same calculation method as the Settings screen. It is also possible to check which context length is being provided when it does not fit in full context, and the reason why the model does not fit.<\/p>\n<h3>Renewal and Major Acceleration of Q8_0 Inference Kernels in ARM CPU Environments<\/h3>\n<p>New kernels have been introduced for ARM CPUs equipped with dot-product instructions, such as AWS Graviton and Ampere, significantly accelerating prompt processing and token generation for Q8_0 models. Linux ARM64 builds automatically detect the presence of these instructions at runtime.<\/p>\n<p>Compared to version 0.2.16, prompt processing is 4 to 13 times faster, and token generation is 1.4 to 5 times faster. In the example shown in the materials, prompt processing for Qwen3.5 4B Q4 in a Graviton4 environment improved from 3.8 tokens\/s to 45 tokens\/s, and token generation improved from 3.0 tokens\/s to 14.7 tokens\/s. Users performing CPU inference on ARM servers can benefit from significant performance improvements simply by updating.<\/p>\n<h3>Improved Memory Estimation Accuracy on NVIDIA and AMD GPUs<\/h3>\n<p>In CUDA and Vulkan environments, pre-estimation processing of memory required for model loading has been improved. In CUDA, instead of securing a fixed 1 GB block, the allocator is changed to calculate only the memory that cannot be reused, and the memory reserved by the driver for kernel execution is now accurately counted. This allows models that were previously limited to 32K on small 8 GB GPUs to be loaded with a full context of up to 128K.<\/p>\n<p>Furthermore, the issue where Mixture-of-Experts (MoE) models consumed several gigabytes more memory than estimated when run on CUDA or Vulkan has been resolved. This is an important improvement for users who want to prevent unexpected out-of-memory (OOM) errors due to insufficient available GPU capacity.<\/p>\n<h3>Deletion of Shared Files in Catalog Models and Fixes for MTP Concurrency<\/h3>\n<p>An issue has been fixed where deleting a specific version of a catalog model incorrectly deleted image files shared with other installed versions, causing images to fail to load. Also, a bug where concurrent requests sent to models drafting with their own MTP (Multi-Token Prediction) head failed with the error &#8220;a causal head in a drafting batch has no selection&#8221; has been resolved.<\/p>\n<p>In addition, processing has been added to prevent error termination and retry if the connection is lost at the start of download, logs for download failure reasons are now output, automatic replacement occurs if a newer release is published while fetching an update, and the corner rounding of the macOS app icon has been fine-tuned.<\/p>\n<h2>Supported Models and Hardware<\/h2>\n<p>This version 0.2.17 includes numerous expansions of supported hardware and optimizations for specific model architectures and quantization formats. You can check the following improvements according to your local environment and the model you want to run.<\/p>\n<h3>Optimized Hardware<\/h3>\n<ul>\n<li><strong>ARM CPUs (AWS Graviton \/ Ampere, etc.)<\/strong>: Optimizations have been made for ARM processors equipped with dot-product instructions. Target processors include the AWS Graviton series (such as Graviton4) and Ampere Computing processors. Linux ARM64 builds incorporate a mechanism to automatically detect whether the CPU supports dot-product instructions at startup. This allows inference to be performed making maximum use of hardware features without manual configuration.<\/li>\n<li><strong>NVIDIA GPUs (CUDA)<\/strong>: The behavior of the memory management allocator and the calculation method for driver-reserved memory in CUDA environments have been revised. Even with relatively small GPUs with 8 GB of VRAM capacity, models that were previously limited to 32K can now be executed with a full context of up to 128K, promoting the effective use of memory resources.<\/li>\n<li><strong>AMD GPUs and Vulkan Environments<\/strong>: Across all environments utilizing the Vulkan backend, the memory footprint estimation for MoE models described below has been fixed, suppressing unexpected excesses in memory consumption.<\/li>\n<\/ul>\n<h3>Supported and Improved Models and Quantization Formats<\/h3>\n<ul>\n<li><strong>Q8_0 Quantization Format<\/strong>: Dedicated kernels have been implemented for ARM CPUs equipped with dot-product instructions, significantly accelerating both prompt processing and token generation for models in Q8_0 format.<\/li>\n<li><strong>Qwen3.5 4B (Q4 Quantization, etc.)<\/strong>: The release notes cite the Q4 model of Qwen3.5 4B as a specific example of operation verification, showing actual measured values where prompt processing in a Graviton4 environment improved from 3.8 tokens\/s to 45 tokens\/s, and generation improved from 3.0 tokens\/s to 14.7 tokens\/s.<\/li>\n<li><strong>Mixture-of-Experts (MoE) Models<\/strong>: The issue in CUDA and Vulkan environments where GPU memory consumption exceeded estimates by several gigabytes during execution has been resolved. Stable loading according to preliminary estimates is possible even for models with large MoE architectures.<\/li>\n<li><strong>MTP (Multi-Token Prediction) Draft-Supported Models<\/strong>: Internal errors that occurred during concurrent request processing in models performing speculative sampling (drafting) utilizing their own MTP heads have been fixed, enabling stable inference processing.<\/li>\n<li><strong>Vision Models (Image Input-Supported Models)<\/strong>: In environments managing multiple versions of catalog models, the handling of shared image files has been improved, enhancing the stability of multimodal inference.<\/li>\n<\/ul>\n<h2>How to Get It<\/h2>\n<p>Specific installation commands for this release are not listed in the materials, so please check the official release page and Magnitude&#8217;s official website when updating to the latest version.<\/p>\n<p>If you are using the Magnitude desktop application, even if a newer release is published while downloading an update, it features a mechanism to automatically detect the new version and replace the older download. Updates can be applied according to the update notification.<\/p>\n<p>For environments using the CLI tool (<code>@magnitudedev\/cli<\/code>) and for detailed introduction procedures and package acquisition on each platform, please refer to the following official resources:<\/p>\n<ul>\n<li>GitHub Releases: <a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/%40magnitudedev\/cli%400.2.17\">magnitudedev\/magnitude @magnitudedev\/cli@0.2.17<\/a><\/li>\n<li>Project Official Website: <a href=\"https:\/\/magnitude.dev\">magnitude.dev<\/a><\/li>\n<\/ul>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/03\/magnitude-cli-v0-2-5-released\/\">Magnitude CLI v0.2.5 Released: M5 Mac &amp; MoE Speedups<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-0-2-16-released\/\">Magnitude v0.2.16 Released: Faster Parallel Requests and Docker<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-14-released\/\">Magnitude CLI 0.2.14 Released: Faster CPU Decoding and More<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-12-release\/\">Magnitude @magnitudedev\/cli@0.2.12 Released<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Follow this tool<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-magnitude-en\/\">Magnitude overview and release history (16 releases tracked)<\/a><\/li>\n<li><strong>Other inference engines and runtimes<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-vllm-en\/\">vLLM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/%40magnitudedev\/cli%400.2.17\">magnitudedev\/magnitude @magnitudedev\/cli@0.2.17 Release Notes<\/a><\/li>\n<li><a href=\"https:\/\/magnitude.dev\">Magnitude Project Official Website<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Version 0.2.17 of the open-source inference engine Magnitude CLI is now available, featuring dynamic context length adjustment and ARM CPU speedups.<\/p>\n","protected":false},"author":1,"featured_media":15604,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[2931,4799,1187,2791,165,1547,511],"class_list":["post-15605","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-magnitudedev-cli-en","tag-arm-en","tag-cuda-en","tag-magnitude-en","tag-moe-en","tag-verified","tag-vulkan-en"],"lang":"en","translations":{"en":15605,"ja":15603},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/15605","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=15605"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/15605\/revisions"}],"predecessor-version":[{"id":18076,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/15605\/revisions\/18076"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/15604"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=15605"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=15605"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=15605"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}