{"id":13529,"date":"2026-11-17T11:02:20","date_gmt":"2026-11-17T02:02:20","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=13529"},"modified":"2026-11-17T11:11:33","modified_gmt":"2026-11-17T02:11:33","slug":"magnitude-cli-0-2-11-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-11-released\/","title":{"rendered":"Magnitude CLI 0.2.11 Released with Custom GGUF Support"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\">magnitudedev\/magnitude<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/@magnitudedev\/cli@0.2.11\">@magnitudedev\/cli@0.2.11<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-26<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>Apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>The latest version, <code>@magnitudedev\/cli@0.2.11<\/code>, has been released for the local inference engine Magnitude. Magnitude is a Rust-based open-source inference engine (Apache-2.0 license) that automatically tunes kernels to match the hardware of the execution environment to achieve fast inference.<\/p>\n<p>The biggest change in this release is the expanded support for running arbitrary custom GGUF models, going beyond just the models registered in the official catalog. In addition to searching and downloading models on Hugging Face, cache files from Ollama and LM Studio that already exist on your PC can now be used as-is without duplication, significantly improving the flexibility of managing models on hand.<\/p>\n<h2>Breaking Changes &amp; Deprecations<\/h2>\n<p>The default sampling parameters have been changed in the OpenAI-compatible API and Anthropic-compatible API. Additionally, the <code>top_k<\/code> parameter is now accepted in the Anthropic-compatible API.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>API<\/th>\n<th>Parameter<\/th>\n<th>Previous Default<\/th>\n<th>New Default<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>OpenAI API<\/td>\n<td><code>temperature<\/code><\/td>\n<td>0.8<\/td>\n<td>1.0<\/td>\n<\/tr>\n<tr>\n<td>OpenAI API<\/td>\n<td><code>top_p<\/code><\/td>\n<td>0.95<\/td>\n<td>0.95<\/td>\n<\/tr>\n<tr>\n<td>OpenAI API<\/td>\n<td><code>top_k<\/code><\/td>\n<td>None (Unspecified)<\/td>\n<td>20<\/td>\n<\/tr>\n<tr>\n<td>Anthropic API<\/td>\n<td><code>temperature<\/code><\/td>\n<td>1.0<\/td>\n<td>1.0<\/td>\n<\/tr>\n<tr>\n<td>Anthropic API<\/td>\n<td><code>top_p<\/code><\/td>\n<td>1.0<\/td>\n<td>0.95<\/td>\n<\/tr>\n<tr>\n<td>Anthropic API<\/td>\n<td><code>top_k<\/code><\/td>\n<td>None (Unsupported)<\/td>\n<td>20<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>This change affects users who use Magnitude&#8217;s compatible API server and do not explicitly specify sampling parameters on the request side. Since omitting parameters may change the diversity or determinism of the output compared to before, please explicitly specify values from the client side as needed.<\/p>\n<h2>Key Changes<\/h2>\n<h3>Support for Custom GGUF Models and Local Cache Auto-Detection<\/h3>\n<p>As long as the model family is supported by Magnitude, you can now run any GGUF outside the catalog (different quantizations, fine-tunes, merged models, uncensored versions, etc.). You can search for and add models on Hugging Face from the desktop app, web app, and CLI (<code>magnitude catalog search<\/code>, <code>inspect<\/code>, <code>pull<\/code>). Before downloading, you can check the model size, whether it can run on your local PC (and the reason if it cannot), and whether it is a gated model. Gated models can be downloaded by signing in to Hugging Face once.<\/p>\n<p>Furthermore, model files in Hugging Face caches, LM Studio, llama.cpp caches, Ollama, and user-added folders on your PC are automatically detected and used in place without duplication. Files can be loaded without being copied, and re-downloading is avoided if the catalog model is already on hand. In addition, the optimal drafter available for each model (its own repository or folder, a matching catalog model, built-in MTP head, etc.) is automatically configured, and can also be changed or disabled.<\/p>\n<h3>Addition of Native Chat Tab<\/h3>\n<p>A native &#8220;Chat&#8221; tab has been added for desktop. It supports streaming output from local models, performance metrics display, reasoning control, generation parameter settings, and saving conversation history using SQLite. In addition to code block language label display, syntax highlighting, and individual copy functions, the display of Markdown headings and nested spaces has also been improved.<\/p>\n<h3>Accelerated Inference Across a Wide Range of Environments<\/h3>\n<p>A number of hardware-specific performance improvements have been implemented. On x86 CPU environments equipped with AVX2 or VNNI instruction sets, execution speed has been improved for k-quant, Q8, and 4-bit codebook weights. On NVIDIA GPU environments, loading of IQ2_XXS has been accelerated, alongside faster decoding for models streaming token tables. According to documentation, the decoding speed of Nemotron 3.5 Bolt (when drafting is disabled) on a GB10 environment has improved from the previous 36.0 tok\/s to 41.9 tok\/s.<\/p>\n<p>Additionally, the time and memory required to prepare model programs on AMD GPUs (Vulkan, RADV) on Linux have been significantly reduced. Speculative decoding (drafting) has been improved to determine behavior based on the actual time taken in the most recent steps, preventing it from stopping on a single delayed step. Furthermore, chat template rendering delays that scanned the entire conversation for each message have been resolved, allowing short response turns to resume from cached conversations.<\/p>\n<h3>Improved Memory Planning Accuracy and Stability<\/h3>\n<p>Model fit checks now accurately calculate backend-specific memory costs. This resolves issues where models marked as runnable failed to load, or where prompt caches for long conversations grew beyond expected sizes. On small GPUs with insufficient VRAM, a message indicating incompatibility is displayed rather than crashing upon load. Also, on PCs equipped with both integrated and discrete GPUs, the discrete GPU is now automatically selected and the device in use is displayed.<\/p>\n<h3>Expansion of Model Catalog and Support for LiLiCorr and NVFP4<\/h3>\n<p>&#8220;Nemotron 3.5 Bolt 74B-A9B&#8221;, &#8220;Nemotron 3.5 Super VL 120B-A12B (Vision)&#8221;, and &#8220;Qwen 3.8 Flash Next NVFP4&#8221; have been added to the catalog. The Nemotron 3.5 models adopt &#8220;LiLiCorr&#8221;, a new drafting method that has been evaluated higher than DFlash2. The decode speed of Bolt in a GB10 environment recorded 55.7 tok\/s (compared to 51.6 tok\/s for llama.cpp). Additionally, NVIDIA&#8217;s NVFP4 models with activation scales can leverage FP4 Tensor cores, achieving a prefill speed of 826 tok\/s on Super VL NVFP4 in a GB10 environment (compared to 552 tok\/s for llama.cpp).<\/p>\n<h3>Individual Model Bug Fixes<\/h3>\n<p>Support was added for reasoning functionality in LFM2.5 Instruct and Thinking models, thought output control in Gemma 4, utilizing base model-derived vision projectors in fine-tuned models, and running Qwen 3.5 models without built-in prediction heads. Furthermore, bugs in sampling probability calculations and weight format compilation errors on the Vulkan backend, as well as weight packing bugs on Metal, have been fixed.<\/p>\n<h2>Supported Models and Hardware<\/h2>\n<p>In this version <code>0.2.11<\/code>, new additions to the model catalog, support for running as custom GGUF models, and backend feature enhancements for various hardware have been implemented.<\/p>\n<h3>Newly Supported Models<\/h3>\n<p>The following models have been newly added to the official catalog. All of them can leverage the latest speculative decoding methods and quantization formats in addition to standard inference features.<\/p>\n<ul>\n<li><strong>Nemotron 3.5 Bolt 74B-A9B<\/strong>: Supports speculative decoding using the new drafting method &#8220;LiLiCorr&#8221; (evaluated higher than DFlash2). In a GB10 environment, it delivers decoding performance of 41.9 tok\/s without drafting and 55.7 tok\/s with LiLiCorr applied.<\/li>\n<li><strong>Nemotron 3.5 Super VL 120B-A12B<\/strong>: A multimodal model equipped with vision capabilities. In addition to normal weight formats, NVFP4 variants are also provided, enabling fast prefill processing on GPUs equipped with FP4 Tensor cores.<\/li>\n<li><strong>Qwen 3.8 Flash Next NVFP4<\/strong>: An NVFP4 format model retaining NVIDIA&#8217;s activation scales.<\/li>\n<\/ul>\n<p>Furthermore, official support has been added to run the following models as custom models (manually added from local files or Hugging Face):<\/p>\n<ul>\n<li><strong>Nemotron 3 Super<\/strong><\/li>\n<li><strong>Nemotron 3 Ultra<\/strong><\/li>\n<li><strong>Qwen 3.5 (Configurations without built-in prediction heads)<\/strong>: Variants lacking a prediction head can now be loaded and run successfully.<\/li>\n<li><strong>LFM2.5 Instruct \/ Thinking<\/strong>: Thought process output (reasoning) now functions correctly.<\/li>\n<li><strong>Gemma 4<\/strong>: Supports on\/off control for thought mode, handling behavior to output thought processes unless explicitly disabled.<\/li>\n<li><strong>Various Fine-Tuned Models<\/strong>: Inference has been made possible for fine-tuned models that repurpose the vision projector belonging to the base model.<\/li>\n<\/ul>\n<h3>Quantization Formats<\/h3>\n<p>Optimizations for supported quantization formats and weight types are progressing.<\/p>\n<ul>\n<li><strong>NVFP4<\/strong>: Added support for NVFP4 models containing NVIDIA activation scales. Native execution is achieved on GPU environments equipped with FP4 Tensor cores, yielding significant prefill speed improvements.<\/li>\n<li><strong>IQ2_XXS<\/strong>: Weight data loading speed has been improved on NVIDIA GPU environments.<\/li>\n<li><strong>k-quant \/ Q8 \/ 4-bit codebook<\/strong>: Processing has been optimized on x86 CPU environments, improving execution speed.<\/li>\n<\/ul>\n<h3>Hardware and Accelerators<\/h3>\n<p>Support and stability for GPUs and CPUs across various platforms have been enhanced.<\/p>\n<ul>\n<li><strong>NVIDIA GPU<\/strong>: Utilization of FP4 Tensor cores in modern architectures like GB10 has progressed, alongside faster decoding for models streaming token tables. Additionally, in PC environments where integrated GPUs (iGPUs) and discrete GPUs (dGPUs) coexist, the discrete GPU is automatically prioritized and assigned, and the name of the device in use is now displayed.<\/li>\n<li><strong>AMD GPU (Linux)<\/strong>: In Vulkan and RADV driver environments, preparation processing for shader programs required for model execution has been overhauled, significantly reducing preparation time and memory consumption. Furthermore, fixes for sampling probability calculations in the Vulkan backend and compilation errors occurring with certain weight formats have been resolved.<\/li>\n<li><strong>Apple Silicon (macOS)<\/strong>: Weight-packing bugs occurring in the Metal backend have been fixed, improving operational stability. In addition, bugs in initial setup processing for macOS have been fixed.<\/li>\n<li><strong>x86 CPU<\/strong>: Inference calculations for various quantization formats have been accelerated in CPU environments supporting AVX2 or VNNI instructions. Also, the issue where CPU inference was occupied for several minutes during large image inputs has been resolved.<\/li>\n<\/ul>\n<h2>How to Update<\/h2>\n<p>This update is distributed as version <code>0.2.11<\/code> of <code>@magnitudedev\/cli<\/code>.<\/p>\n<p>While the documentation does not directly list the update command, if you are using the desktop app or CLI environment, please obtain the latest version through the official distribution page or your environment&#8217;s package management tool. Note that an issue where existing applications failed to properly replace when updating from previous versions on Windows 10 environments has been fixed in this version, allowing for safe upgrades on Windows as well.<\/p>\n<p>For details on the latest installers and binaries for each platform, please check the following official release page:<\/p>\n<ul>\n<li>GitHub Releases: <a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/%40magnitudedev%2Fcli%400.2.11\">magnitudedev\/magnitude @magnitudedev\/cli@0.2.11<\/a><\/li>\n<\/ul>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-10-released-2\/\">Magnitude CLI 0.2.10 Released with Memory Fixes and New Quantizations<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-14-released\/\">Magnitude CLI 0.2.14 Released: Faster CPU Decoding and More<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-9-released\/\">Magnitude @magnitudedev\/cli@0.2.9 Released<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/magnitude-cli-0-2-8-released\/\">Magnitude CLI 0.2.8 Released with VRAM Fixes and Gemma 4 Support<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Follow this tool<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-magnitude-en\/\">Magnitude overview and release history (16 releases tracked)<\/a><\/li>\n<li><strong>Other inference engines and runtimes<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-vllm-en\/\">vLLM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/%40magnitudedev%2Fcli%400.2.11\">magnitudedev\/magnitude @magnitudedev\/cli@0.2.11<\/a><\/li>\n<\/ul>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/%40magnitudedev\/cli%400.2.11\">https:\/\/github.com\/magnitudedev\/magnitude\/releases\/tag\/%40magnitudedev\/cli%400.2.11<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Magnitude 0.2.11 brings custom GGUF model execution, local cache auto-detection, a native Chat tab, and speedups across various hardware.<\/p>\n","protected":false},"author":1,"featured_media":13528,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[1634,163,2791,695,2793,1547],"class_list":["post-13529","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-amd-gpu-en","tag-gguf-en","tag-magnitude-en","tag-nvidia-en","tag-rust-en","tag-verified"],"lang":"en","translations":{"en":13529,"ja":13527},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13529","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=13529"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13529\/revisions"}],"predecessor-version":[{"id":17940,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/13529\/revisions\/17940"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/13528"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=13529"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=13529"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=13529"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}