vllm-d3 GGUF Models Released: Decision Models for Local Run

At a Glance
| Item | Value |
|---|---|
| Repository | ggml-org/vllm-d3-GGUF |
| Family guide | Qwen3.8 guide (10 articles) |
| Publisher guide | Alibaba (Qwen): models and licenses |
| Published | 2026-10-12 |
| License | apache-2.0 |
| Formats | GGUF |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
ggml-org has released GGUF format conversions of the “vllm-d3" series of decision models. Five models have been published: “ggml-org/vllm-d3-GGUF", along with “vllm-d3-nano-GGUF", “vllm-d3-mini-GGUF", “vllm-d3-lite-GGUF", and “vllm-d3-flash-GGUF".
These models are GGUF conversions of the vLLM Semantic Router Decision 3.0 model group using an automated conversion tool. They take text, JSON, images, or video as input, and are specialized for decision-making tasks—returning determination results and probabilities for choices, Yes/No, and scores in a single call without generating text. They are designed to be easily executed in local environments via llama.cpp’s /v1/systemone API and other interfaces.
Specifications
The specifications listed in the model card for the original model “vllm-sr/d3-flash" are as follows:
- Parameters: 8.39B (including a 0.46B vision encoder)
- Architecture: Qwen3_5Model
Performance
Since the newly released repositories are GGUF conversions, benchmark results from the original model “vllm-sr/d3-flash"'s model card are presented below. Quantized versions may exhibit slight accuracy variations due to the quantization process.
Below is the comparison table for the “Jev Decision Index 0.3.1", which evaluates text-input decision-making performance. The figures are taken directly from the publisher’s model card.
→ Scroll horizontally to see all columns
| Model | Jev Decision Index ↑ | Public ↑ | Same-skill tests ↑ | New-domain tasks ↑ |
|---|---|---|---|---|
| d3-flash | 59.0 | 57.8 | 57.5 | 52.9 |
| Bespoke Nimble 9B v3 | 54.7 | 57.2 | 52.3 | 49.1 |
| Nace.AI Drex v1.5 | 50.8 | 58.1 | 51.9 | 39.7 |
| AJev lora5 (Gemma 4 12B) | 50.0 | 51.7 | 51.4 | 41.5 |
| Winnow-12B | 49.3 | 50.7 | 48.1 | 44.2 |
| Decision 2.0 (9B) | 48.2 | 46.8 | 47.4 | 44.5 |
According to measurements by the publisher, d3-flash recorded an overall score of 59.0 on the Jev Decision Index, significantly improving upon the 48.2 of the previous-generation Decision 2.0 (9B). It also marks the highest values in overall scores—including Same-skill tests (57.5) and New-domain tasks (52.9)—compared to 9B and 12B models in the same scale category. However, in the Public-only score, it fell slightly short of Nace.AI Drex v1.5’s 58.1, remaining at 57.8.
Next is the comparison table for the “Jev Decision Index vision board 0.3.1", which shows decision-making performance for image inputs.
| Model | Vision Index ↑ | Public ↑ | Private ↑ |
|---|---|---|---|
| d3-flash | 66.9 | 69.5 | 64.3 |
| JPT-9B | 61.7 | 65.4 | 58.0 |
| Winnow-12B | 58.8 | 63.9 | 53.6 |
| Jev-Omni (12B) | 56.1 | 60.1 | 52.1 |
In image input evaluations as well, d3-flash scores 66.9 on the Vision Index, 69.5 on Public, and 64.3 on Private, outperforming other models such as JPT-9B and Winnow-12B.
Additionally, the following measured values for image and video benchmarks of d3-flash alone have been published. The MMMU-Pro vision image understanding indicator stands at 38.7.
| Benchmark | d3-flash |
|---|---|
| CV-Bench | 75.2 |
| BLINK | 55.5 |
| RealWorldQA | 63.4 |
| CharXiv † | 83.0 |
| InfographicVQA † | 95.1 |
| Mind2Web † | 85.1 |
| Winoground | 83.3 |
| KIE (CORD+FUNSD) † | 96.3 |
| Moderation (Hateful Memes) | 41.3 |
| R-Bench-M | 20.8 |
| MMMU-Pro vision | 38.7 |
Furthermore, the results for the Perception Test (validation), a video understanding benchmark, are as follows (internal evaluation values by the publisher):
| Perception Test (validation) | d3-flash |
|---|---|
| All 19,140 questions | 73.3 |
| Memory | 75.5 |
| Abstraction | 68.7 |
| Physics | 66.1 |
| Semantics | 86.1 |
The overall score for all 19,140 questions is 73.3. By category, it shows high accuracy in Semantics at 86.1 and Memory at 75.5, while Physics stands at 66.1.
Strengths and Use Cases
Unlike conversational models that perform general text generation, the Decision 3.0 series, starting with the original model “vllm-sr/d3-flash", is designed as a “Decision Model" that directly performs structured judgments on inputs.
Inputs can simultaneously include text and JSON, as well as images and video. A single request can process multiple images (in PNG, JPEG, and WebP formats) or videos sampled at 2 frames per second (in MP4, WebM, MOV, and MKV formats), answering multiple questions about them in a single inference pass. Supported question formats include three types: “Choice" from multiple alternatives, binary “Yes / No", and stepped evaluation “Score". Because probability values for each choice are output directly, there is no need to parse output text.
Expected use cases include routing user inquiries to appropriate departments, document processing to determine the presence of required items in images such as receipts and invoices, and multimodal verification tasks to identify object movements or state changes in video. Additionally, taking advantage of its property of not generating text, it is well-suited for rapid classification and filtering in the early stages of pipeline processing.
How It Differs from Similar Models
Previously, our site covered the “clef" structured decision model: 12GB+ required VRAM / GGUF version available, which also specializes in structured judgments. While Clef is a 27B-scale multimodal decision model based on Qwen3.8-27B, the vllm-d3 series is based on various Qwen3.5 models. It differs in that users can choose from a wide range of model sizes to suit their use cases and compute resources, including nano (based on 2B), mini (based on 4B), lite (based on 0.8B), flash (based on 9B), and a standard version based on 27B.
Moreover, whereas “OrcaSAQ-2-27B" text generation model: answer examples on our site / required VRAM 16GB+, another Qwen3.8-27B derivative model, is a generative model handling regular dialogue and text generation, this series is decisively different in its design, specializing in the “System One" API to return choices and probabilities directly without generating text.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 26.1B parameters (taken from the base model vllm-sr/d3)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 24GB (RTX 4090 / 3090, etc.) | Q4_K_M | 18.1GB | 21.7GB |
| 32GB (RTX 5090, etc.) | Q8_0 | 26.6GB | 32.0GB |
| 80GB class (A100 / H100) | BF16 | 50.1GB | 60.1GB |
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Runs in Ollama, LM Studio and llama.cpp as-is.
It is distributed in GGUF, so no conversion is needed.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compression: the Q4_K_M build measures 5.96 bits per weight — about 37% the size of the original 16-bit weights, calculated by this site from the actual file sizes.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with 15–40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| ggml-org/Muse-Glimmer-30B-GGUF | 29.8B | 24GB | apache-2.0 | Muse-Glimmer-30B-GGUF 30B Multimodal Local Agent Model: 24GB+ VRAM (2026-11-17) |
| Cloudflare/clef-omni | 35.3B | 80GB | apache-2.0 | clef-omni 30B Multimodal Decision Model: 80GB+ VRAM (2026-11-17) |
| inclusionAI/ZwZ2-27B-GGUF | 27.4B | 24GB | — | ZwZ2-27B-GGUF Vision-Language Model: 24GB+ VRAM (2026-11-17) |
| ggml-org/OpenJev-GGUF | 27.4B | 24GB | cc-by-nc-4.0 | OpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM (2026-10-01) |
How to Get It
The models released this time are distributed in GGUF format and can be obtained from their respective Hugging Face repositories. Since they are not gated models, they can be downloaded directly without applying for agreement to individual terms of use.
To run a model using llama.cpp’s server functionality, execute the following command:
llama serve -hf ggml-org/vllm-d3-GGUF
When running a smaller model, specify the repository name. For example, to run d3-flash:
llama serve -hf ggml-org/vllm-d3-flash-GGUF
After startup, you can send API requests via the /v1/systemone endpoint to execute decision-making processes. It also supports llama.app running in the browser.
Related Articles
- Signal-3.8-27B-GGUF: Our Test Answers, 16GB+ VRAM
- Qwen3.8-27B-DSpine-b16 Draft Model for Speculative Decoding: 8GB+ VRAM
- Qwen3-Coder-30B-A3B-Instruct-GGUF Text Generation Model: 24GB+ VRAM
- OpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM
What to Read Next
- Find models by VRAM (This model runs from the 24GB tier) → Other models that run on a 24GB GPU
- Explore the same model family → Qwen3.8 family overview (10 articles, 17 converted builds)
- Engines that run this model → llama.cpp / Ollama
- What Q4_K_M, Q8_0 mean and where to get this model → GGUF format guide and models / Quantization and model-format glossary
- Learn about the publisher → Alibaba (Qwen): models, licenses and articles
Sources
- https://huggingface.co/ggml-org/vllm-d3-GGUF
- https://huggingface.co/ggml-org/vllm-d3-nano-GGUF
- https://huggingface.co/ggml-org/vllm-d3-mini-GGUF
- https://huggingface.co/ggml-org/vllm-d3-lite-GGUF
- https://huggingface.co/ggml-org/vllm-d3-flash-GGUF
- https://huggingface.co/vllm-sr/d3-flash
- https://github.com/ggml-org/llama.cpp/pull/30383

