TheDrummer_Orion-26B-A4B-v1.1-GGUF Multimodal Model: 12GB+ VRAM

October 1, 2026

TheDrummer_Orion-26B-A4B-v1.1-GGUF Multimodal Model: 12GB+ VRAM

At a Glance

Item Value
Repository bartowski/TheDrummer_Orion-26B-A4B-v1.1-GGUF
Published 2026-09-14
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

What we checked ourselves

  • It needs about 18% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.22× as much Japanese fits in the same context length, and generation is faster per character.

Details and conditions are in “Our Own Measurements” below.

Overview

It is reported that bartowski has released GGUF quantization files for the multimodal model Orion-26B-A4B-v1.1. As some information included could not be officially verified at the time of writing, caution is advised when deploying or testing the model in practice. It supports text, image, and audio inputs, and a variety of quantization files are provided for local execution.

Specifications

  • Parameters: 26B
  • Input Support: Text, image, audio (requires mmproj file)
  • Speculative Decoding: None
  • imatrix: Supported
  • Architecture: Gemma4ForConditionalGeneration

Performance

Regarding the performance of the base model, according to the model card, a low temperature setting (such as 0.9) is optimal, and Gemma’s thinking capabilities are reported to be very sharp in roleplay (RP). Below is a comparison table of per-tensor layouts for the published quantized versions.

→ Scroll horizontally to see all columns

Quant Size Body bits/weight File bits/weight Body kept at base type
Q6_K_L 24.52GB 7.38 7.60 51 %
Q6_K 23.79GB 7.03 7.37 71 %
Q6_K_S 23.14GB 6.71 7.17 90 %
Q5_K_M 20.01GB 5.92 6.20 71 %
Q5_K_S 18.40GB 5.70 5.70 90 %
Q4_K_L 18.21GB 5.19 5.64 50 %
Q4_K_M 17.97GB 5.07 5.57 70 %
Q4_K_S 16.05GB 4.70 4.98 90 %
IQ4_NL 15.63GB 5.07 4.85 70 %
Q3_K_L 14.80GB 4.39 4.59 50 %
IQ3_M 14.34GB 4.17 4.45 50 %
IQ4_XS 14.28GB 4.42 4.43 90 %
Q3_K_M 13.79GB 4.01 4.27 70 %
IQ3_XS 12.84GB 3.57 3.98 90 %
IQ3_XXS 12.61GB 3.46 3.91 70 %
Q3_K_S 12.37GB 3.56 3.84 90 %
Q2_K 11.61GB 3.19 3.60 70 %
IQ2_M 10.90GB 2.89 3.38 70 %
IQ2_S 10.31GB 2.61 3.20 70 %
IQ2_XS 9.90GB 2.41 3.07 90 %

(Table truncated. Subsequent rows omitted)

Furthermore, the comparative verification results between the computed layout and the standard layout are shown below.

→ Scroll horizontally to see all columns

Quant Computed layout KLD Standard layout KLD Ratio at equal size Size vs standard file
Q4_K_M 0.8086 ± 0.0113 1.1462 ± 0.0132 0.88× +5.5 %
Q3_K_M 1.2642 ± 0.0149 2.9859 ± 0.0241 0.51× +5.9 %
IQ2_XS 2.6730 ± 0.0206 2.9355 ± 0.0210 0.91× −2.3 %

From these tables, it is confirmed that quantization files adopting the computed layout, such as Q4_K_M and Q3_K_M, keep the Kullback-Leibler divergence (KLD) lower compared to the standard layout, resulting in smaller differences from the unquantized model and maintained quality. Q4_K_M in particular shows a favorable balance.

Strengths and Use Cases

This model supports image and audio input in addition to text, and is intended for conversational use and roleplay (RP) applications. Inheriting the characteristics of the base model, it is well-suited for interactive tasks leveraging multimodal inputs.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 25.8B parameters (taken from the base model TheDrummer/Orion-26B-A4B-v1.1)

Your VRAM Quantization File size Est. memory needed
12GB (RTX 4070 / 3060 12GB, etc.) IQ2_S 9.6GB 11.5GB
16GB (RTX 5060 Ti 16GB / 4060 Ti 16GB, etc.) IQ3_M 13.4GB 16.0GB
24GB (RTX 4090 / 3090, etc.) Q5_K_M 18.6GB 22.4GB
32GB (RTX 5090, etc.) Q8_0 25.0GB 30.0GB
80GB class (A100 / H100) BF16 47.0GB 56.4GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

Compression: the IQ2_S build measures 3.2 bits per weight — about 20% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists.

Our Own Measurements

Values we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model — not figures copied from the model card. How we measure, and the results for every model: Our Measurements.

Measurements We Did Not Take

This model does not state a license, so we cannot confirm permission to use it. Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.

Japanese Token Efficiency

Tokenizer Tokens per 1,000 Japanese characters Ratio to the same text in English
This model 564 1.03×
Qwen3 688 1.26×
Llama 3.2 744 1.36×
Gemma 3 564 1.03×
gpt-oss 795 1.45×
LLM-jp-3 497 0.85×

It needs about 18% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.22× as much Japanese fits in the same context length, and generation is faster per character.

Counted with the tokenizer.json of TheDrummer/Orion-26B-A4B-v1.1 on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.

Inside the GGUF File

File: TheDrummer_Orion-26B-A4B-v1.1-Q4_K_M.gguf (16.73GB, Q4_K_M). We read only the header (metadata) of the file, not the weights.

Item Value
Architecture (as named in the GGUF) gemma4
Maximum trained context length 262,144 tokens
Layers 30
Experts 8 active out of 128
Vocabulary size 262,144
Chat template Included (mentions tool calls, has a thinking switch)
imatrix Used (TheDrummer_Orion-26B-A4B-v1.1-calibration-v6.txt, 492 chunks)
Weight types (share of parameters) Q4_K 52.3% / Q8_0 20.2% / Q5_0 16.5% / Q6_K 9.0% / other 2.1%
Average bits per weight 5.69 bits
Embedding / output layer type Q6_K / shared with the embedding

The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.

How to Get It

  • Distribution Format: GGUF
  • Supported Engines: llama.cpp, LM Studio, koboldcpp, ramalama, Jan AI, Text Generation Web UI, LoLLMs, Atomic Chat
  • Example Download Command:
hf download bartowski/TheDrummer_Orion-26B-A4B-v1.1-GGUF --include "TheDrummer_Orion-26B-A4B-v1.1-Q4_K_M.gguf" --local-dir./

When utilizing multimodal inputs, you will also need to prepare the corresponding multimodal projector file (mmproj-TheDrummer_Orion-26B-A4B-v1.1-f16.gguf or mmproj-TheDrummer_Orion-26B-A4B-v1.1-bf16.gguf).

Related Articles

What to Read Next

Sources

Update History

  • 2026-09-15: Verified the content against the official primary source.
  • 2026-09-24: Rebuilt the article: Added a “can you run it locally?" verdict from the distribution formats, converted builds, engine registries, license terms and measured compression
  • 2026-09-25: Changed the title to show what the article covers (VRAM requirements, file list, etc.).
  • 2026-09-28: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.

New Modelsbartowski,GGUF,llama.cpp,Orion-26B-A4B-v1.1,マルチモーダル,確認済み