{"id":21655,"date":"2026-11-21T06:10:47","date_gmt":"2026-11-20T21:10:47","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/11\/21\/inkling-2-mini-gguf-released\/"},"modified":"2026-11-22T06:47:27","modified_gmt":"2026-11-21T21:47:27","slug":"inkling-2-mini-gguf-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/21\/inkling-2-mini-gguf-released\/","title":{"rendered":"Inkling 2 Mini GGUF Released: Looped MoE Multimodal Model"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/ggml-org\/Inkling-2-Mini-GGUF\">ggml-org\/Inkling-2-Mini-GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Family guide<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-thinkingmachines-inkling-2-en\/\">Inkling-2 guide (1 articles)<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-11-20<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<p><!-- lmw:lab-summary --><\/p>\n<p><strong>What we checked ourselves<\/strong><\/p>\n<ul>\n<li>It needs about 17% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.85\u00d7 as much Japanese fits in the same context length.<\/li>\n<\/ul>\n<p>Details and conditions are in \u201cOur Own Measurements\u201d below.<\/p>\n<p><!-- \/lmw:lab-summary --><\/p>\n<h2>Overview<\/h2>\n<p>GGUF quantized versions of the multimodal language model &#8220;Inkling 2 Mini&#8221;, developed by Thinking Machines Lab, have been released by ggml-org and AtomicChat. Inkling 2 Mini is a looped mixture-of-experts (MoE) model that accepts text and image inputs, designed with local inference in mind. It features a unique structure that repeatedly evaluates intermediate layers.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Parameters: 26.4B total parameters, 3.4B active parameters (effectively 5.4B with loop execution)<\/li>\n<li>Architecture: MoE (128 total experts, 8 active experts per token), looped transformer (24-layer structure, where the middle 12 layers are executed in a loop 3 times, effectively evaluating 48 layers)<\/li>\n<li>Context Length: Up to 1M tokens<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>The evaluation results published in the original model card (<code>thinkingmachines\/Inkling-2-Mini<\/code>) are as follows. These figures are measurement results from the original model, and accuracy may vary in the quantized version. The comparison table lists all model columns from the original data as-is.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th><\/th>\n<th>Inkling 2 Mini<\/th>\n<th>Gemma 4 31B<\/th>\n<th>Muse Glimmer<\/th>\n<th>Qwen3.8-27B<\/th>\n<th>Nemotron 3.5 Super VL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Model Info<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Params (B) activated \/ total<\/td>\n<td>5\/26<\/td>\n<td>31\/31<\/td>\n<td>30\/30<\/td>\n<td>27\/27<\/td>\n<td>12\/125<\/td>\n<\/tr>\n<tr>\n<td>Third Party Index<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>AA Index (v5.0)<\/td>\n<td>20.1<\/td>\n<td>4.4<\/td>\n<td>6.3<\/td>\n<td>20.1<\/td>\n<td>13.6<\/td>\n<\/tr>\n<tr>\n<td>Vals Index (v2.2)<\/td>\n<td>46.5%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>29.8%<\/td>\n<\/tr>\n<tr>\n<td>Coding and Autoresearch<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>FrontierCode v2.0 stirrup v0.2.1<\/td>\n<td>15.1%<\/td>\n<td>0.0%<\/td>\n<td>1.5%<\/td>\n<td>14.9%<\/td>\n<td>6.1%<\/td>\n<\/tr>\n<tr>\n<td>DeepSWE v1.1 mini-swe-agent v2.4.6<\/td>\n<td>48.2%<\/td>\n<td>0.3%<\/td>\n<td>4.4%<\/td>\n<td>42.2%<\/td>\n<td>24.7%<\/td>\n<\/tr>\n<tr>\n<td>Terminal Bench v4.0 mini-swe-agent v2.4.6<\/td>\n<td>5.6%<\/td>\n<td>0.0%<\/td>\n<td>0.5%<\/td>\n<td>5.6%<\/td>\n<td>0.0%<\/td>\n<\/tr>\n<tr>\n<td>Terminal Bench v2.1 Inkling 2 Mini in pi v0.83.0<\/td>\n<td>71.6%<\/td>\n<td>43.4%<\/td>\n<td>51.7%<\/td>\n<td>73.0%<\/td>\n<td>60.7%<\/td>\n<\/tr>\n<tr>\n<td>SWEBench Pro v2 (Hard) mini-swe-agent v2.4.6<\/td>\n<td>77.9%<\/td>\n<td>14.2%<\/td>\n<td>44.3%<\/td>\n<td>73.8%<\/td>\n<td>44.1%<\/td>\n<\/tr>\n<tr>\n<td>SWEBench Multilingual Inkling 2 Mini with bash only<\/td>\n<td>73.7%<\/td>\n<td>43.0%<\/td>\n<td>66.9%<\/td>\n<td>70.0%<\/td>\n<td>67.7%<\/td>\n<\/tr>\n<tr>\n<td>SWEBench Verified bash only<\/td>\n<td>79.2%<\/td>\n<td>46.1%<\/td>\n<td>76.0%<\/td>\n<td>77.8%<\/td>\n<td>71.1%<\/td>\n<\/tr>\n<tr>\n<td>Vibe Code Bench v1.1 openhands (vals)<\/td>\n<td>77.9%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>64.9%<\/td>\n<td>3.5%<\/td>\n<\/tr>\n<tr>\n<td>PMPP-Hard (AUC, 0\u201310h) pi v0.83.0<\/td>\n<td>455.82<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>262.50<\/td>\n<td>\u2013<\/td>\n<\/tr>\n<tr>\n<td>Agentic Search, Tool Use, and Knowledge Work<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>BrowseComp Inkling 2 Mini with compaction<\/td>\n<td>79.0%<\/td>\n<td>37.0%<\/td>\n<td>63.9%<\/td>\n<td>61.6%<\/td>\n<td>78.9%<\/td>\n<\/tr>\n<tr>\n<td>DeepSearchQA Inkling 2 Mini with compaction<\/td>\n<td>85.3%<\/td>\n<td>61.7%<\/td>\n<td>74.6%<\/td>\n<td>79.7%<\/td>\n<td>77.7%<\/td>\n<\/tr>\n<tr>\n<td>MCP Atlas Public<\/td>\n<td>82.7%<\/td>\n<td>54.2%<\/td>\n<td>75.5%<\/td>\n<td>77.3%<\/td>\n<td>67.7%<\/td>\n<\/tr>\n<tr>\n<td>AA-Briefcase v1.1<\/td>\n<td>1263<\/td>\n<td>369<\/td>\n<td>478<\/td>\n<td>1391<\/td>\n<td>999<\/td>\n<\/tr>\n<tr>\n<td>GDPVal-AA v2.1<\/td>\n<td>1319<\/td>\n<td>627<\/td>\n<td>792<\/td>\n<td>1427<\/td>\n<td>1162<\/td>\n<\/tr>\n<tr>\n<td>JobBench<\/td>\n<td>49.6%<\/td>\n<td>8.2%<\/td>\n<td>16.1%<\/td>\n<td>33.4%<\/td>\n<td>36.5%<\/td>\n<\/tr>\n<tr>\n<td>AutomationBench-AA v1.0.6<\/td>\n<td>41.8%<\/td>\n<td>4.9%<\/td>\n<td>6.8%<\/td>\n<td>48.2%<\/td>\n<td>14.8%<\/td>\n<\/tr>\n<tr>\n<td>EnterpriseOps-Gym<\/td>\n<td>42.0%<\/td>\n<td>28.4%<\/td>\n<td>34.7%<\/td>\n<td>44.2%<\/td>\n<td>28.1%<\/td>\n<\/tr>\n<tr>\n<td>Vals Finance Index<\/td>\n<td>50.1%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>41.0%<\/td>\n<\/tr>\n<tr>\n<td>AA-AnalystAgent<\/td>\n<td>46.3%<\/td>\n<td>\u2013<\/td>\n<td>20.0%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<\/tr>\n<tr>\n<td>Diligence Bench Best harness<\/td>\n<td>48.6%<\/td>\n<td>\u2013<\/td>\n<td>40.9%<\/td>\n<td>46.4%<\/td>\n<td>45.0%<\/td>\n<\/tr>\n<tr>\n<td>Vision Interpretation and Creation<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>GDP.pdf<\/td>\n<td>16.8%<\/td>\n<td>6.0%<\/td>\n<td>11.8%<\/td>\n<td>16.4%<\/td>\n<td>8.0%<\/td>\n<\/tr>\n<tr>\n<td>PerceptionBench<\/td>\n<td>57.5%<\/td>\n<td>40.7%<\/td>\n<td>42.8%<\/td>\n<td>52.4%<\/td>\n<td>36.8%<\/td>\n<\/tr>\n<tr>\n<td>MMMU Pro Standard 10<\/td>\n<td>74.9%<\/td>\n<td>73.4%<\/td>\n<td>74.3%<\/td>\n<td>76.3%<\/td>\n<td>72.3%<\/td>\n<\/tr>\n<tr>\n<td>OmniDocBench v1.6<\/td>\n<td>92.8%<\/td>\n<td>82.5%<\/td>\n<td>90.4%<\/td>\n<td>91.5%<\/td>\n<td>77.9%<\/td>\n<\/tr>\n<tr>\n<td>ScreenSpot Pro original \/ with Python<\/td>\n<td>76.5\/87.7%<\/td>\n<td>5.9%\/\u2013<\/td>\n<td>75.4%\/\u2013<\/td>\n<td>57.6\/78.9%<\/td>\n<td>43.5%\/\u2013<\/td>\n<\/tr>\n<tr>\n<td>Chartography original \/ with Python<\/td>\n<td>22.1\/72.5%<\/td>\n<td>12.0\/11.8%<\/td>\n<td>17.4\/25.1%<\/td>\n<td>12.2\/52.1%<\/td>\n<td>9.7\/29.3%<\/td>\n<\/tr>\n<tr>\n<td>ZeroBench (main) original \/ with Python<\/td>\n<td>17.0\/34.0%<\/td>\n<td>9.0\/12.0%<\/td>\n<td>9.0\/17.0%<\/td>\n<td>12.0%\/\u2013<\/td>\n<td>11.0\/12.0%<\/td>\n<\/tr>\n<tr>\n<td>MuirBench<\/td>\n<td>77.5%<\/td>\n<td>72.3%<\/td>\n<td>77.6%<\/td>\n<td>65.9%<\/td>\n<td>72.7%<\/td>\n<\/tr>\n<tr>\n<td>BVB<\/td>\n<td>68.4%<\/td>\n<td>47.0%<\/td>\n<td>50.5%<\/td>\n<td>70.7%<\/td>\n<td>52.1%<\/td>\n<\/tr>\n<tr>\n<td>Reasoning, Knowledge, and Epistemics<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>HLE-Diamond<\/td>\n<td>15.2%<\/td>\n<td>7.2%<\/td>\n<td>9.1%<\/td>\n<td>15.3%<\/td>\n<td>8.8%<\/td>\n<\/tr>\n<tr>\n<td>CritPt v2 Full \/ Hard<\/td>\n<td>31.4\/17.0%<\/td>\n<td>\u2013\/4.0%<\/td>\n<td>\u2013\/7.0%<\/td>\n<td>10.6\/4.0%<\/td>\n<td>36.9\/13.5%<\/td>\n<\/tr>\n<tr>\n<td>Apex 2025<\/td>\n<td>34.4%<\/td>\n<td>5.2%<\/td>\n<td>4.2%<\/td>\n<td>18.8%<\/td>\n<td>22.9%<\/td>\n<\/tr>\n<tr>\n<td>ARC-AGI v1 \/ v2<\/td>\n<td>84.5\/49.3%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>87.5\/42.4%<\/td>\n<td>22.7\/1.5%<\/td>\n<\/tr>\n<tr>\n<td>LongBench v2<\/td>\n<td>66.9%<\/td>\n<td>52.9%<\/td>\n<td>33.1%<\/td>\n<td>52.3%<\/td>\n<td>62.1%<\/td>\n<\/tr>\n<tr>\n<td>GraphWalks BFS 256K<\/td>\n<td>69.7%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>50.4%<\/td>\n<td>63.3%<\/td>\n<\/tr>\n<tr>\n<td>MRCR v2 8-needle 256K\u2013512K<\/td>\n<td>46.4%<\/td>\n<td>\u2013<\/td>\n<td>\u2013<\/td>\n<td>38.0%<\/td>\n<td>5.7%<\/td>\n<\/tr>\n<tr>\n<td>AA-Omniscience v1.1 score<\/td>\n<td>-8.6<\/td>\n<td>-48.5<\/td>\n<td>-32.2<\/td>\n<td>-6.4<\/td>\n<td>-2.9<\/td>\n<\/tr>\n<tr>\n<td>AA-Omniscience v1.1 non-hallucination rate<\/td>\n<td>63.4%<\/td>\n<td>14.5%<\/td>\n<td>16.4%<\/td>\n<td>71.4%<\/td>\n<td>69.5%<\/td>\n<\/tr>\n<tr>\n<td>ComplexConstraints v1.1 openhands v1.45.0<\/td>\n<td>33.4%<\/td>\n<td>31.6%<\/td>\n<td>35.6%<\/td>\n<td>30.1%<\/td>\n<td>30.1%<\/td>\n<\/tr>\n<tr>\n<td>Safety and Alignment<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>FORTRESS (adversarial) Public<\/td>\n<td>89.5%<\/td>\n<td>67.8%<\/td>\n<td>90.3%<\/td>\n<td>78.8%<\/td>\n<td>79.6%<\/td>\n<\/tr>\n<tr>\n<td>FORTRESS (benign) Public<\/td>\n<td>96.4%<\/td>\n<td>97.6%<\/td>\n<td>89.1%<\/td>\n<td>97.8%<\/td>\n<td>92.7%<\/td>\n<\/tr>\n<tr>\n<td>StrongREJECT (none)<\/td>\n<td>97.8%<\/td>\n<td>98.1%<\/td>\n<td>99.0%<\/td>\n<td>99.0%<\/td>\n<td>99.4%<\/td>\n<\/tr>\n<tr>\n<td>Misbehavior (Internal) \u2193<\/td>\n<td>9.8%<\/td>\n<td>27.6%<\/td>\n<td>46.4%<\/td>\n<td>57.7%<\/td>\n<td>52.8%<\/td>\n<\/tr>\n<tr>\n<td>Covert misbehavior (Internal) \u2193<\/td>\n<td>8.3%<\/td>\n<td>24.3%<\/td>\n<td>45.6%<\/td>\n<td>33.2%<\/td>\n<td>43.4%<\/td>\n<\/tr>\n<tr>\n<td>Destructive actions (Internal) \u2193<\/td>\n<td>10.2%<\/td>\n<td>75.8%<\/td>\n<td>68.8%<\/td>\n<td>35.2%<\/td>\n<td>65.6%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>According to measurements by the publishers, this model achieves high evaluations in SWE-bench (79.2% on SWEBench Verified), which measures real-world GitHub problem-solving capability, and LongBench v2 (66.9%), which measures long-context understanding, demonstrating superiority over comparative open-weight models. It also scores 15.2% on HLE-Diamond, which handles expert-level ultra-difficult problems, delivering top-class results. In MMMU (74.9% on MMMU Pro Standard 10), which measures the ability to solve university-level problems with images, it maintains a level comparable to Qwen3.8-27B (76.3%). On the other hand, in the evaluation on Terminal-Bench v2.1 (71.6%), there are items where it falls slightly short of Qwen3.8-27B (73.0%).<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>According to the original model design and model card description, Inkling 2 Mini is a multimodal model capable of processing text and image inputs, making it suitable for the following use cases:<\/p>\n<ul>\n<li><strong>Agents and Tool-Use Systems<\/strong>: Suitable for building tasks involving command execution and external tool operation.<\/li>\n<li><strong>Coding Assistants<\/strong>: Well-suited for code writing, debugging, and software engineering-related real-world problem solving.<\/li>\n<li><strong>Chatbots and RAG Systems<\/strong>: Supports a 1M token long context, making it useful for reading long texts and general conversational processing.<\/li>\n<li><strong>Visual Tasks<\/strong>: Because it processes images and text in a shared latent space, it can handle image-based interpretation and response generation.<\/li>\n<\/ul>\n<p>Additionally, the depth of reasoning during inference can be controlled via the <code>reasoning_effort<\/code> parameter. Available settings include <code>none<\/code>, <code>minimal<\/code>, <code>low<\/code>, <code>medium<\/code>, <code>high<\/code> (default), <code>xhigh<\/code>, and <code>max<\/code>, as well as numerical values from 0.0 to 0.99.<\/p>\n<p>In terms of language, it supports English and general multilingual use, though output quality may decrease in languages other than English.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>24GB (RTX 4090 \/ 3090, etc.)<\/td>\n<td>IQ4_NL<\/td>\n<td>16.4GB<\/td>\n<td>19.7GB<\/td>\n<\/tr>\n<tr>\n<td>32GB (RTX 5090, etc.)<\/td>\n<td>Q8_0<\/td>\n<td>25.9GB<\/td>\n<td>31.1GB<\/td>\n<\/tr>\n<tr>\n<td>80GB class (A100 \/ H100)<\/td>\n<td>BF16<\/td>\n<td>48.6GB<\/td>\n<td>58.4GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p><strong>Runs in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\n<p>It is distributed in GGUF, so no conversion is needed.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats, converted builds we have found, and each engine&#8217;s own model registry. &#8220;Not found&#8221; means we have not seen such a build, not that none exists.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:lab --><\/p>\n<h2>Our Own Measurements<\/h2>\n<p>Values we checked ourselves on our server (no GPU) by actually reading this model&#8217;s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/observations-en\/\">Our Measurements<\/a>.<\/p>\n<h3>Measurements We Did Not Take<\/h3>\n<p>This model does not state a license, so we cannot confirm permission to use it. Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\n<h3>Japanese Token Efficiency<\/h3>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Tokenizer<\/th>\n<th>Tokens per 1,000 Japanese characters<\/th>\n<th>Ratio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>This model<\/strong><\/td>\n<td><strong>806<\/strong><\/td>\n<td><strong>1.38\u00d7<\/strong><\/td>\n<\/tr>\n<tr>\n<td>Qwen3<\/td>\n<td>688<\/td>\n<td>1.26\u00d7<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.2<\/td>\n<td>744<\/td>\n<td>1.36\u00d7<\/td>\n<\/tr>\n<tr>\n<td>Gemma 3<\/td>\n<td>564<\/td>\n<td>1.03\u00d7<\/td>\n<\/tr>\n<tr>\n<td>gpt-oss<\/td>\n<td>795<\/td>\n<td>1.45\u00d7<\/td>\n<\/tr>\n<tr>\n<td>LLM-jp-3<\/td>\n<td>497<\/td>\n<td>0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>It needs about 17% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.85\u00d7 as much Japanese fits in the same context length.<\/p>\n<p>Counted with the <code>tokenizer.json<\/code> of <a href=\"https:\/\/huggingface.co\/thinkingmachines\/Inkling-2-Mini\">thinkingmachines\/Inkling-2-Mini<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<h3>Inside the GGUF File<\/h3>\n<p>File: <a href=\"https:\/\/huggingface.co\/ggml-org\/Inkling-2-Mini-GGUF\/blob\/main\/Inkling-2-Mini-IQ4_NL.gguf\">Inkling-2-Mini-IQ4_NL.gguf<\/a> (16.42GB, <code>IQ4_NL<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Architecture (as named in the GGUF)<\/td>\n<td><code>inkling2_loop<\/code><\/td>\n<\/tr>\n<tr>\n<td>Maximum trained context length<\/td>\n<td>1,048,576 tokens<\/td>\n<\/tr>\n<tr>\n<td>Layers<\/td>\n<td>24<\/td>\n<\/tr>\n<tr>\n<td>Experts<\/td>\n<td>8 active out of 128<\/td>\n<\/tr>\n<tr>\n<td>Vocabulary size<\/td>\n<td>200,058<\/td>\n<\/tr>\n<tr>\n<td>Chat template<\/td>\n<td>Included (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n<tr>\n<td>imatrix<\/td>\n<td>Used (<code>inkling2-loop-calibration-v6.txt<\/code>, 1,770 chunks)<\/td>\n<\/tr>\n<tr>\n<td>Weight types (share of parameters)<\/td>\n<td>IQ4_NL 78.2% \/ Q8_0 21.7% \/ F32 0.1%<\/td>\n<\/tr>\n<tr>\n<td>Average bits per weight<\/td>\n<td>5.41 bits<\/td>\n<\/tr>\n<tr>\n<td>Embedding \/ output layer type<\/td>\n<td>Q8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<p><!-- \/lmw:lab --><\/p>\n<h2>How to Get It<\/h2>\n<p>GGUF format models of Inkling 2 Mini are distributed in the ggml-org and AtomicChat repositories on Hugging Face. Like the original model, it is released under the Apache 2.0 license and does not require prior agreement to terms of use (gated).<\/p>\n<p>The quantized version published in the ggml-org repository can be run with the following command:<\/p>\n<pre><code class=\"language-bash\">llama serve -hf ggml-org\/Inkling-2-Mini-GGUF:IQ4_NL\n<\/code><\/pre>\n<p>Note that because this model employs a special architecture that loops intermediate layers, running it with llama.cpp requires a build that includes pull requests supporting the loop structure (such as <code>ggml-org\/llama.cpp#32695<\/code>, commit <code>6196d98<\/code>). Regular llama.cpp releases without the corresponding code will not be able to load the model.<\/p>\n<p><!-- lmw:variants --><\/p>\n<h2>Quantized and Converted Variants<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Added<\/th>\n<th>Publisher<\/th>\n<th>Format<\/th>\n<th>Repository<\/th>\n<th>Smallest VRAM tier (build, est. memory)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>2026-11-21<\/td>\n<td>lmstudio-community<\/td>\n<td>GGUF<\/td>\n<td><a href=\"https:\/\/huggingface.co\/lmstudio-community\/Inkling-2-Mini-GGUF\">lmstudio-community\/Inkling-2-Mini-GGUF<\/a><\/td>\n<td>Q4_K_M 19.5GB (fits in 24GB VRAM)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>File sizes of each build:<\/p>\n<ul>\n<li>Available builds in lmstudio-community\/Inkling-2-Mini-GGUF: Q4_K_M 16.3GB \/ Q6_K 21.9GB \/ Q8_0 25.9GB<\/li>\n<\/ul>\n<p><em>This section is appended automatically by Local Model Watch when a converted build of this model appears after publication. Memory figures are estimated from the size of the distributed files. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:variants --><\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/21\/tml-d24-varloop-research-gguf-2\/\">TML-d24-varloop-research-GGUF Vision-Language Model: 24GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/qwen3-coder-30b-a3b-instruct-gguf-2\/\">Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/vllm-d3-gguf-models-released\/\">vllm-d3 GGUF Models Released: Decision Models for Local Run<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/muse-glimmer-30b-gguf-2\/\">Muse-Glimmer-30B-GGUF 30B Multimodal Local Agent Model: 24GB+ VRAM<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 24GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-24gb-en\/\">Other models that run on a 24GB GPU<\/a><\/li>\n<li><strong>Explore the same model family<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/model-thinkingmachines-inkling-2-en\/\">Inkling-2 family overview (1 articles, 1 converted builds)<\/a><\/li>\n<li><strong>Engines that run this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a><\/li>\n<li><strong>What IQ4_NL, Q8_0 mean and where to get this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-gguf-en\/\">GGUF format guide and models<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">Quantization and model-format glossary<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-vision\">Other vision-language models<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/ggml-org\/Inkling-2-Mini-GGUF\">https:\/\/huggingface.co\/ggml-org\/Inkling-2-Mini-GGUF<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/AtomicChat\/Inkling-2-Mini-GGUF\">https:\/\/huggingface.co\/AtomicChat\/Inkling-2-Mini-GGUF<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/thinkingmachines\/Inkling-2-Mini\">https:\/\/huggingface.co\/thinkingmachines\/Inkling-2-Mini<\/a><\/li>\n<\/ul>\n<p><!-- lmw:updates --><\/p>\n<h2>Update History<\/h2>\n<ul>\n<li>2026-11-21: Added converted builds to \u201cQuantized and Converted Variants\u201d: lmstudio-community\/Inkling-2-Mini-GGUF<\/li>\n<li>2026-11-22: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<p><!-- \/lmw:updates --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>GGUF quantized versions of Inkling 2 Mini, a looped MoE multimodal language model by Thinking Machines Lab, are now available.<\/p>\n","protected":false},"author":1,"featured_media":21654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,163,5509,165,5511,1547,1952,5513],"class_list":["post-21655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-gguf-en","tag-inkling-2-mini-en","tag-moe-en","tag-thinking-machines-lab-en","tag-verified","tag-vlm-en","tag--en"],"lang":"en","translations":{"en":21655,"ja":21653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/21655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=21655"}],"version-history":[{"count":4,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/21655\/revisions"}],"predecessor-version":[{"id":22205,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/21655\/revisions\/22205"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/21654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=21655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=21655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=21655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}