Inkling 2 Mini GGUF Released: Looped MoE Multimodal Model

November 22, 2026

Inkling 2 Mini GGUF Released: Looped MoE Multimodal Model

At a Glance

Item Value
Repository ggml-org/Inkling-2-Mini-GGUF
Family guide Inkling-2 guide (1 articles)
Published 2026-11-20
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

What we checked ourselves

  • It needs about 17% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.85× as much Japanese fits in the same context length.

Details and conditions are in “Our Own Measurements” below.

Overview

GGUF quantized versions of the multimodal language model “Inkling 2 Mini", developed by Thinking Machines Lab, have been released by ggml-org and AtomicChat. Inkling 2 Mini is a looped mixture-of-experts (MoE) model that accepts text and image inputs, designed with local inference in mind. It features a unique structure that repeatedly evaluates intermediate layers.

Specifications

  • Parameters: 26.4B total parameters, 3.4B active parameters (effectively 5.4B with loop execution)
  • Architecture: MoE (128 total experts, 8 active experts per token), looped transformer (24-layer structure, where the middle 12 layers are executed in a loop 3 times, effectively evaluating 48 layers)
  • Context Length: Up to 1M tokens

Performance

The evaluation results published in the original model card (thinkingmachines/Inkling-2-Mini) are as follows. These figures are measurement results from the original model, and accuracy may vary in the quantized version. The comparison table lists all model columns from the original data as-is.

→ Scroll horizontally to see all columns

Inkling 2 Mini Gemma 4 31B Muse Glimmer Qwen3.8-27B Nemotron 3.5 Super VL
Model Info
Params (B) activated / total 5/26 31/31 30/30 27/27 12/125
Third Party Index
AA Index (v5.0) 20.1 4.4 6.3 20.1 13.6
Vals Index (v2.2) 46.5% – – – 29.8%
Coding and Autoresearch
FrontierCode v2.0 stirrup v0.2.1 15.1% 0.0% 1.5% 14.9% 6.1%
DeepSWE v1.1 mini-swe-agent v2.4.6 48.2% 0.3% 4.4% 42.2% 24.7%
Terminal Bench v4.0 mini-swe-agent v2.4.6 5.6% 0.0% 0.5% 5.6% 0.0%
Terminal Bench v2.1 Inkling 2 Mini in pi v0.83.0 71.6% 43.4% 51.7% 73.0% 60.7%
SWEBench Pro v2 (Hard) mini-swe-agent v2.4.6 77.9% 14.2% 44.3% 73.8% 44.1%
SWEBench Multilingual Inkling 2 Mini with bash only 73.7% 43.0% 66.9% 70.0% 67.7%
SWEBench Verified bash only 79.2% 46.1% 76.0% 77.8% 71.1%
Vibe Code Bench v1.1 openhands (vals) 77.9% – – 64.9% 3.5%
PMPP-Hard (AUC, 0–10h) pi v0.83.0 455.82 – – 262.50 –
Agentic Search, Tool Use, and Knowledge Work
BrowseComp Inkling 2 Mini with compaction 79.0% 37.0% 63.9% 61.6% 78.9%
DeepSearchQA Inkling 2 Mini with compaction 85.3% 61.7% 74.6% 79.7% 77.7%
MCP Atlas Public 82.7% 54.2% 75.5% 77.3% 67.7%
AA-Briefcase v1.1 1263 369 478 1391 999
GDPVal-AA v2.1 1319 627 792 1427 1162
JobBench 49.6% 8.2% 16.1% 33.4% 36.5%
AutomationBench-AA v1.0.6 41.8% 4.9% 6.8% 48.2% 14.8%
EnterpriseOps-Gym 42.0% 28.4% 34.7% 44.2% 28.1%
Vals Finance Index 50.1% – – – 41.0%
AA-AnalystAgent 46.3% – 20.0% – –
Diligence Bench Best harness 48.6% – 40.9% 46.4% 45.0%
Vision Interpretation and Creation
GDP.pdf 16.8% 6.0% 11.8% 16.4% 8.0%
PerceptionBench 57.5% 40.7% 42.8% 52.4% 36.8%
MMMU Pro Standard 10 74.9% 73.4% 74.3% 76.3% 72.3%
OmniDocBench v1.6 92.8% 82.5% 90.4% 91.5% 77.9%
ScreenSpot Pro original / with Python 76.5/87.7% 5.9%/– 75.4%/– 57.6/78.9% 43.5%/–
Chartography original / with Python 22.1/72.5% 12.0/11.8% 17.4/25.1% 12.2/52.1% 9.7/29.3%
ZeroBench (main) original / with Python 17.0/34.0% 9.0/12.0% 9.0/17.0% 12.0%/– 11.0/12.0%
MuirBench 77.5% 72.3% 77.6% 65.9% 72.7%
BVB 68.4% 47.0% 50.5% 70.7% 52.1%
Reasoning, Knowledge, and Epistemics
HLE-Diamond 15.2% 7.2% 9.1% 15.3% 8.8%
CritPt v2 Full / Hard 31.4/17.0% –/4.0% –/7.0% 10.6/4.0% 36.9/13.5%
Apex 2025 34.4% 5.2% 4.2% 18.8% 22.9%
ARC-AGI v1 / v2 84.5/49.3% – – 87.5/42.4% 22.7/1.5%
LongBench v2 66.9% 52.9% 33.1% 52.3% 62.1%
GraphWalks BFS 256K 69.7% – – 50.4% 63.3%
MRCR v2 8-needle 256K–512K 46.4% – – 38.0% 5.7%
AA-Omniscience v1.1 score -8.6 -48.5 -32.2 -6.4 -2.9
AA-Omniscience v1.1 non-hallucination rate 63.4% 14.5% 16.4% 71.4% 69.5%
ComplexConstraints v1.1 openhands v1.45.0 33.4% 31.6% 35.6% 30.1% 30.1%
Safety and Alignment
FORTRESS (adversarial) Public 89.5% 67.8% 90.3% 78.8% 79.6%
FORTRESS (benign) Public 96.4% 97.6% 89.1% 97.8% 92.7%
StrongREJECT (none) 97.8% 98.1% 99.0% 99.0% 99.4%
Misbehavior (Internal) ↓ 9.8% 27.6% 46.4% 57.7% 52.8%
Covert misbehavior (Internal) ↓ 8.3% 24.3% 45.6% 33.2% 43.4%
Destructive actions (Internal) ↓ 10.2% 75.8% 68.8% 35.2% 65.6%

According to measurements by the publishers, this model achieves high evaluations in SWE-bench (79.2% on SWEBench Verified), which measures real-world GitHub problem-solving capability, and LongBench v2 (66.9%), which measures long-context understanding, demonstrating superiority over comparative open-weight models. It also scores 15.2% on HLE-Diamond, which handles expert-level ultra-difficult problems, delivering top-class results. In MMMU (74.9% on MMMU Pro Standard 10), which measures the ability to solve university-level problems with images, it maintains a level comparable to Qwen3.8-27B (76.3%). On the other hand, in the evaluation on Terminal-Bench v2.1 (71.6%), there are items where it falls slightly short of Qwen3.8-27B (73.0%).

Strengths and Use Cases

According to the original model design and model card description, Inkling 2 Mini is a multimodal model capable of processing text and image inputs, making it suitable for the following use cases:

  • Agents and Tool-Use Systems: Suitable for building tasks involving command execution and external tool operation.
  • Coding Assistants: Well-suited for code writing, debugging, and software engineering-related real-world problem solving.
  • Chatbots and RAG Systems: Supports a 1M token long context, making it useful for reading long texts and general conversational processing.
  • Visual Tasks: Because it processes images and text in a shared latent space, it can handle image-based interpretation and response generation.

Additionally, the depth of reasoning during inference can be controlled via the reasoning_effort parameter. Available settings include none, minimal, low, medium, high (default), xhigh, and max, as well as numerical values from 0.0 to 0.99.

In terms of language, it supports English and general multilingual use, though output quality may decrease in languages other than English.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch)

Your VRAM Quantization File size Est. memory needed
24GB (RTX 4090 / 3090, etc.) IQ4_NL 16.4GB 19.7GB
32GB (RTX 5090, etc.) Q8_0 25.9GB 31.1GB
80GB class (A100 / H100) BF16 48.6GB 58.4GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists.

Our Own Measurements

Values we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model — not figures copied from the model card. How we measure, and the results for every model: Our Measurements.

Measurements We Did Not Take

This model does not state a license, so we cannot confirm permission to use it. Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.

Japanese Token Efficiency

Tokenizer Tokens per 1,000 Japanese characters Ratio to the same text in English
This model 806 1.38×
Qwen3 688 1.26×
Llama 3.2 744 1.36×
Gemma 3 564 1.03×
gpt-oss 795 1.45×
LLM-jp-3 497 0.85×

It needs about 17% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.85× as much Japanese fits in the same context length.

Counted with the tokenizer.json of thinkingmachines/Inkling-2-Mini on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.

Inside the GGUF File

File: Inkling-2-Mini-IQ4_NL.gguf (16.42GB, IQ4_NL). We read only the header (metadata) of the file, not the weights.

Item Value
Architecture (as named in the GGUF) inkling2_loop
Maximum trained context length 1,048,576 tokens
Layers 24
Experts 8 active out of 128
Vocabulary size 200,058
Chat template Included (mentions tool calls, has a thinking switch)
imatrix Used (inkling2-loop-calibration-v6.txt, 1,770 chunks)
Weight types (share of parameters) IQ4_NL 78.2% / Q8_0 21.7% / F32 0.1%
Average bits per weight 5.41 bits
Embedding / output layer type Q8_0 / Q8_0

The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.

How to Get It

GGUF format models of Inkling 2 Mini are distributed in the ggml-org and AtomicChat repositories on Hugging Face. Like the original model, it is released under the Apache 2.0 license and does not require prior agreement to terms of use (gated).

The quantized version published in the ggml-org repository can be run with the following command:

llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL

Note that because this model employs a special architecture that loops intermediate layers, running it with llama.cpp requires a build that includes pull requests supporting the loop structure (such as ggml-org/llama.cpp#32695, commit 6196d98). Regular llama.cpp releases without the corresponding code will not be able to load the model.

Quantized and Converted Variants

→ Scroll horizontally to see all columns

Added Publisher Format Repository Smallest VRAM tier (build, est. memory)
2026-11-21 lmstudio-community GGUF lmstudio-community/Inkling-2-Mini-GGUF Q4_K_M 19.5GB (fits in 24GB VRAM)

File sizes of each build:

  • Available builds in lmstudio-community/Inkling-2-Mini-GGUF: Q4_K_M 16.3GB / Q6_K 21.9GB / Q8_0 25.9GB

This section is appended automatically by Local Model Watch when a converted build of this model appears after publication. Memory figures are estimated from the size of the distributed files. What the quantization names mean: glossary.

Related Articles

What to Read Next

Sources

Update History

  • 2026-11-21: Added converted builds to “Quantized and Converted Variants”: lmstudio-community/Inkling-2-Mini-GGUF
  • 2026-11-22: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.