Inkling 2 Mini GGUF Released: Looped MoE Multimodal Model

At a Glance
| Item | Value |
|---|---|
| Repository | ggml-org/Inkling-2-Mini-GGUF |
| Family guide | Inkling-2 guide (1 articles) |
| Published | 2026-11-20 |
| Formats | GGUF |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
What we checked ourselves
- It needs about 17% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.85× as much Japanese fits in the same context length.
Details and conditions are in “Our Own Measurements” below.
Overview
GGUF quantized versions of the multimodal language model “Inkling 2 Mini", developed by Thinking Machines Lab, have been released by ggml-org and AtomicChat. Inkling 2 Mini is a looped mixture-of-experts (MoE) model that accepts text and image inputs, designed with local inference in mind. It features a unique structure that repeatedly evaluates intermediate layers.
Specifications
- Parameters: 26.4B total parameters, 3.4B active parameters (effectively 5.4B with loop execution)
- Architecture: MoE (128 total experts, 8 active experts per token), looped transformer (24-layer structure, where the middle 12 layers are executed in a loop 3 times, effectively evaluating 48 layers)
- Context Length: Up to 1M tokens
Performance
The evaluation results published in the original model card (thinkingmachines/Inkling-2-Mini) are as follows. These figures are measurement results from the original model, and accuracy may vary in the quantized version. The comparison table lists all model columns from the original data as-is.
→ Scroll horizontally to see all columns
| Inkling 2 Mini | Gemma 4 31B | Muse Glimmer | Qwen3.8-27B | Nemotron 3.5 Super VL | |
|---|---|---|---|---|---|
| Model Info | |||||
| Params (B) activated / total | 5/26 | 31/31 | 30/30 | 27/27 | 12/125 |
| Third Party Index | |||||
| AA Index (v5.0) | 20.1 | 4.4 | 6.3 | 20.1 | 13.6 |
| Vals Index (v2.2) | 46.5% | – | – | – | 29.8% |
| Coding and Autoresearch | |||||
| FrontierCode v2.0 stirrup v0.2.1 | 15.1% | 0.0% | 1.5% | 14.9% | 6.1% |
| DeepSWE v1.1 mini-swe-agent v2.4.6 | 48.2% | 0.3% | 4.4% | 42.2% | 24.7% |
| Terminal Bench v4.0 mini-swe-agent v2.4.6 | 5.6% | 0.0% | 0.5% | 5.6% | 0.0% |
| Terminal Bench v2.1 Inkling 2 Mini in pi v0.83.0 | 71.6% | 43.4% | 51.7% | 73.0% | 60.7% |
| SWEBench Pro v2 (Hard) mini-swe-agent v2.4.6 | 77.9% | 14.2% | 44.3% | 73.8% | 44.1% |
| SWEBench Multilingual Inkling 2 Mini with bash only | 73.7% | 43.0% | 66.9% | 70.0% | 67.7% |
| SWEBench Verified bash only | 79.2% | 46.1% | 76.0% | 77.8% | 71.1% |
| Vibe Code Bench v1.1 openhands (vals) | 77.9% | – | – | 64.9% | 3.5% |
| PMPP-Hard (AUC, 0–10h) pi v0.83.0 | 455.82 | – | – | 262.50 | – |
| Agentic Search, Tool Use, and Knowledge Work | |||||
| BrowseComp Inkling 2 Mini with compaction | 79.0% | 37.0% | 63.9% | 61.6% | 78.9% |
| DeepSearchQA Inkling 2 Mini with compaction | 85.3% | 61.7% | 74.6% | 79.7% | 77.7% |
| MCP Atlas Public | 82.7% | 54.2% | 75.5% | 77.3% | 67.7% |
| AA-Briefcase v1.1 | 1263 | 369 | 478 | 1391 | 999 |
| GDPVal-AA v2.1 | 1319 | 627 | 792 | 1427 | 1162 |
| JobBench | 49.6% | 8.2% | 16.1% | 33.4% | 36.5% |
| AutomationBench-AA v1.0.6 | 41.8% | 4.9% | 6.8% | 48.2% | 14.8% |
| EnterpriseOps-Gym | 42.0% | 28.4% | 34.7% | 44.2% | 28.1% |
| Vals Finance Index | 50.1% | – | – | – | 41.0% |
| AA-AnalystAgent | 46.3% | – | 20.0% | – | – |
| Diligence Bench Best harness | 48.6% | – | 40.9% | 46.4% | 45.0% |
| Vision Interpretation and Creation | |||||
| GDP.pdf | 16.8% | 6.0% | 11.8% | 16.4% | 8.0% |
| PerceptionBench | 57.5% | 40.7% | 42.8% | 52.4% | 36.8% |
| MMMU Pro Standard 10 | 74.9% | 73.4% | 74.3% | 76.3% | 72.3% |
| OmniDocBench v1.6 | 92.8% | 82.5% | 90.4% | 91.5% | 77.9% |
| ScreenSpot Pro original / with Python | 76.5/87.7% | 5.9%/– | 75.4%/– | 57.6/78.9% | 43.5%/– |
| Chartography original / with Python | 22.1/72.5% | 12.0/11.8% | 17.4/25.1% | 12.2/52.1% | 9.7/29.3% |
| ZeroBench (main) original / with Python | 17.0/34.0% | 9.0/12.0% | 9.0/17.0% | 12.0%/– | 11.0/12.0% |
| MuirBench | 77.5% | 72.3% | 77.6% | 65.9% | 72.7% |
| BVB | 68.4% | 47.0% | 50.5% | 70.7% | 52.1% |
| Reasoning, Knowledge, and Epistemics | |||||
| HLE-Diamond | 15.2% | 7.2% | 9.1% | 15.3% | 8.8% |
| CritPt v2 Full / Hard | 31.4/17.0% | –/4.0% | –/7.0% | 10.6/4.0% | 36.9/13.5% |
| Apex 2025 | 34.4% | 5.2% | 4.2% | 18.8% | 22.9% |
| ARC-AGI v1 / v2 | 84.5/49.3% | – | – | 87.5/42.4% | 22.7/1.5% |
| LongBench v2 | 66.9% | 52.9% | 33.1% | 52.3% | 62.1% |
| GraphWalks BFS 256K | 69.7% | – | – | 50.4% | 63.3% |
| MRCR v2 8-needle 256K–512K | 46.4% | – | – | 38.0% | 5.7% |
| AA-Omniscience v1.1 score | -8.6 | -48.5 | -32.2 | -6.4 | -2.9 |
| AA-Omniscience v1.1 non-hallucination rate | 63.4% | 14.5% | 16.4% | 71.4% | 69.5% |
| ComplexConstraints v1.1 openhands v1.45.0 | 33.4% | 31.6% | 35.6% | 30.1% | 30.1% |
| Safety and Alignment | |||||
| FORTRESS (adversarial) Public | 89.5% | 67.8% | 90.3% | 78.8% | 79.6% |
| FORTRESS (benign) Public | 96.4% | 97.6% | 89.1% | 97.8% | 92.7% |
| StrongREJECT (none) | 97.8% | 98.1% | 99.0% | 99.0% | 99.4% |
| Misbehavior (Internal) ↓ | 9.8% | 27.6% | 46.4% | 57.7% | 52.8% |
| Covert misbehavior (Internal) ↓ | 8.3% | 24.3% | 45.6% | 33.2% | 43.4% |
| Destructive actions (Internal) ↓ | 10.2% | 75.8% | 68.8% | 35.2% | 65.6% |
According to measurements by the publishers, this model achieves high evaluations in SWE-bench (79.2% on SWEBench Verified), which measures real-world GitHub problem-solving capability, and LongBench v2 (66.9%), which measures long-context understanding, demonstrating superiority over comparative open-weight models. It also scores 15.2% on HLE-Diamond, which handles expert-level ultra-difficult problems, delivering top-class results. In MMMU (74.9% on MMMU Pro Standard 10), which measures the ability to solve university-level problems with images, it maintains a level comparable to Qwen3.8-27B (76.3%). On the other hand, in the evaluation on Terminal-Bench v2.1 (71.6%), there are items where it falls slightly short of Qwen3.8-27B (73.0%).
Strengths and Use Cases
According to the original model design and model card description, Inkling 2 Mini is a multimodal model capable of processing text and image inputs, making it suitable for the following use cases:
- Agents and Tool-Use Systems: Suitable for building tasks involving command execution and external tool operation.
- Coding Assistants: Well-suited for code writing, debugging, and software engineering-related real-world problem solving.
- Chatbots and RAG Systems: Supports a 1M token long context, making it useful for reading long texts and general conversational processing.
- Visual Tasks: Because it processes images and text in a shared latent space, it can handle image-based interpretation and response generation.
Additionally, the depth of reasoning during inference can be controlled via the reasoning_effort parameter. Available settings include none, minimal, low, medium, high (default), xhigh, and max, as well as numerical values from 0.0 to 0.99.
In terms of language, it supports English and general multilingual use, though output quality may decrease in languages other than English.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 24GB (RTX 4090 / 3090, etc.) | IQ4_NL | 16.4GB | 19.7GB |
| 32GB (RTX 5090, etc.) | Q8_0 | 25.9GB | 31.1GB |
| 80GB class (A100 / H100) | BF16 | 48.6GB | 58.4GB |
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Runs in Ollama, LM Studio and llama.cpp as-is.
It is distributed in GGUF, so no conversion is needed.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists.
Our Own Measurements
Values we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model — not figures copied from the model card. How we measure, and the results for every model: Our Measurements.
Measurements We Did Not Take
This model does not state a license, so we cannot confirm permission to use it. Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.
Japanese Token Efficiency
| Tokenizer | Tokens per 1,000 Japanese characters | Ratio to the same text in English |
|---|---|---|
| This model | 806 | 1.38× |
| Qwen3 | 688 | 1.26× |
| Llama 3.2 | 744 | 1.36× |
| Gemma 3 | 564 | 1.03× |
| gpt-oss | 795 | 1.45× |
| LLM-jp-3 | 497 | 0.85× |
It needs about 17% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.85× as much Japanese fits in the same context length.
Counted with the tokenizer.json of thinkingmachines/Inkling-2-Mini on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.
Inside the GGUF File
File: Inkling-2-Mini-IQ4_NL.gguf (16.42GB, IQ4_NL). We read only the header (metadata) of the file, not the weights.
| Item | Value |
|---|---|
| Architecture (as named in the GGUF) | inkling2_loop |
| Maximum trained context length | 1,048,576 tokens |
| Layers | 24 |
| Experts | 8 active out of 128 |
| Vocabulary size | 200,058 |
| Chat template | Included (mentions tool calls, has a thinking switch) |
| imatrix | Used (inkling2-loop-calibration-v6.txt, 1,770 chunks) |
| Weight types (share of parameters) | IQ4_NL 78.2% / Q8_0 21.7% / F32 0.1% |
| Average bits per weight | 5.41 bits |
| Embedding / output layer type | Q8_0 / Q8_0 |
The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.
How to Get It
GGUF format models of Inkling 2 Mini are distributed in the ggml-org and AtomicChat repositories on Hugging Face. Like the original model, it is released under the Apache 2.0 license and does not require prior agreement to terms of use (gated).
The quantized version published in the ggml-org repository can be run with the following command:
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL
Note that because this model employs a special architecture that loops intermediate layers, running it with llama.cpp requires a build that includes pull requests supporting the loop structure (such as ggml-org/llama.cpp#32695, commit 6196d98). Regular llama.cpp releases without the corresponding code will not be able to load the model.
Quantized and Converted Variants
→ Scroll horizontally to see all columns
| Added | Publisher | Format | Repository | Smallest VRAM tier (build, est. memory) |
|---|---|---|---|---|
| 2026-11-21 | lmstudio-community | GGUF | lmstudio-community/Inkling-2-Mini-GGUF | Q4_K_M 19.5GB (fits in 24GB VRAM) |
File sizes of each build:
- Available builds in lmstudio-community/Inkling-2-Mini-GGUF: Q4_K_M 16.3GB / Q6_K 21.9GB / Q8_0 25.9GB
This section is appended automatically by Local Model Watch when a converted build of this model appears after publication. Memory figures are estimated from the size of the distributed files. What the quantization names mean: glossary.
Related Articles
- TML-d24-varloop-research-GGUF Vision-Language Model: 24GB+ VRAM
- Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM
- vllm-d3 GGUF Models Released: Decision Models for Local Run
- Muse-Glimmer-30B-GGUF 30B Multimodal Local Agent Model: 24GB+ VRAM
What to Read Next
- Find models by VRAM (This model runs from the 24GB tier) → Other models that run on a 24GB GPU
- Explore the same model family → Inkling-2 family overview (1 articles, 1 converted builds)
- Engines that run this model → llama.cpp / Ollama
- What IQ4_NL, Q8_0 mean and where to get this model → GGUF format guide and models / Quantization and model-format glossary
- Other models for the same task → Other vision-language models
Sources
- https://huggingface.co/ggml-org/Inkling-2-Mini-GGUF
- https://huggingface.co/AtomicChat/Inkling-2-Mini-GGUF
- https://huggingface.co/thinkingmachines/Inkling-2-Mini
Update History
- 2026-11-21: Added converted builds to “Quantized and Converted Variants”: lmstudio-community/Inkling-2-Mini-GGUF
- 2026-11-22: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.

