vllm-d3 GGUF Models Released: Decision Models for Local Run

vllm-d3 GGUF Models Released: Decision Models for Local Run

At a Glance

Item Value
Repository ggml-org/vllm-d3-GGUF
Family guide Qwen3.8 guide (10 articles)
Publisher guide Alibaba (Qwen): models and licenses
Published 2026-10-12
License apache-2.0
Formats GGUF
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

ggml-org has released GGUF format conversions of the “vllm-d3" series of decision models. Five models have been published: “ggml-org/vllm-d3-GGUF", along with “vllm-d3-nano-GGUF", “vllm-d3-mini-GGUF", “vllm-d3-lite-GGUF", and “vllm-d3-flash-GGUF".

These models are GGUF conversions of the vLLM Semantic Router Decision 3.0 model group using an automated conversion tool. They take text, JSON, images, or video as input, and are specialized for decision-making tasks—returning determination results and probabilities for choices, Yes/No, and scores in a single call without generating text. They are designed to be easily executed in local environments via llama.cpp’s /v1/systemone API and other interfaces.

Specifications

The specifications listed in the model card for the original model “vllm-sr/d3-flash" are as follows:

  • Parameters: 8.39B (including a 0.46B vision encoder)
  • Architecture: Qwen3_5Model

Performance

Since the newly released repositories are GGUF conversions, benchmark results from the original model “vllm-sr/d3-flash"'s model card are presented below. Quantized versions may exhibit slight accuracy variations due to the quantization process.

Below is the comparison table for the “Jev Decision Index 0.3.1", which evaluates text-input decision-making performance. The figures are taken directly from the publisher’s model card.

→ Scroll horizontally to see all columns

Model Jev Decision Index ↑ Public ↑ Same-skill tests ↑ New-domain tasks ↑
d3-flash 59.0 57.8 57.5 52.9
Bespoke Nimble 9B v3 54.7 57.2 52.3 49.1
Nace.AI Drex v1.5 50.8 58.1 51.9 39.7
AJev lora5 (Gemma 4 12B) 50.0 51.7 51.4 41.5
Winnow-12B 49.3 50.7 48.1 44.2
Decision 2.0 (9B) 48.2 46.8 47.4 44.5

According to measurements by the publisher, d3-flash recorded an overall score of 59.0 on the Jev Decision Index, significantly improving upon the 48.2 of the previous-generation Decision 2.0 (9B). It also marks the highest values in overall scores—including Same-skill tests (57.5) and New-domain tasks (52.9)—compared to 9B and 12B models in the same scale category. However, in the Public-only score, it fell slightly short of Nace.AI Drex v1.5’s 58.1, remaining at 57.8.

Next is the comparison table for the “Jev Decision Index vision board 0.3.1", which shows decision-making performance for image inputs.

Model Vision Index ↑ Public ↑ Private ↑
d3-flash 66.9 69.5 64.3
JPT-9B 61.7 65.4 58.0
Winnow-12B 58.8 63.9 53.6
Jev-Omni (12B) 56.1 60.1 52.1

In image input evaluations as well, d3-flash scores 66.9 on the Vision Index, 69.5 on Public, and 64.3 on Private, outperforming other models such as JPT-9B and Winnow-12B.

Additionally, the following measured values for image and video benchmarks of d3-flash alone have been published. The MMMU-Pro vision image understanding indicator stands at 38.7.

Benchmark d3-flash
CV-Bench 75.2
BLINK 55.5
RealWorldQA 63.4
CharXiv † 83.0
InfographicVQA † 95.1
Mind2Web † 85.1
Winoground 83.3
KIE (CORD+FUNSD) † 96.3
Moderation (Hateful Memes) 41.3
R-Bench-M 20.8
MMMU-Pro vision 38.7

Furthermore, the results for the Perception Test (validation), a video understanding benchmark, are as follows (internal evaluation values by the publisher):

Perception Test (validation) d3-flash
All 19,140 questions 73.3
Memory 75.5
Abstraction 68.7
Physics 66.1
Semantics 86.1

The overall score for all 19,140 questions is 73.3. By category, it shows high accuracy in Semantics at 86.1 and Memory at 75.5, while Physics stands at 66.1.

Strengths and Use Cases

Unlike conversational models that perform general text generation, the Decision 3.0 series, starting with the original model “vllm-sr/d3-flash", is designed as a “Decision Model" that directly performs structured judgments on inputs.

Inputs can simultaneously include text and JSON, as well as images and video. A single request can process multiple images (in PNG, JPEG, and WebP formats) or videos sampled at 2 frames per second (in MP4, WebM, MOV, and MKV formats), answering multiple questions about them in a single inference pass. Supported question formats include three types: “Choice" from multiple alternatives, binary “Yes / No", and stepped evaluation “Score". Because probability values for each choice are output directly, there is no need to parse output text.

Expected use cases include routing user inquiries to appropriate departments, document processing to determine the presence of required items in images such as receipts and invoices, and multimodal verification tasks to identify object movements or state changes in video. Additionally, taking advantage of its property of not generating text, it is well-suited for rapid classification and filtering in the early stages of pipeline processing.

How It Differs from Similar Models

Previously, our site covered the “clef" structured decision model: 12GB+ required VRAM / GGUF version available, which also specializes in structured judgments. While Clef is a 27B-scale multimodal decision model based on Qwen3.8-27B, the vllm-d3 series is based on various Qwen3.5 models. It differs in that users can choose from a wide range of model sizes to suit their use cases and compute resources, including nano (based on 2B), mini (based on 4B), lite (based on 0.8B), flash (based on 9B), and a standard version based on 27B.

Moreover, whereas “OrcaSAQ-2-27B" text generation model: answer examples on our site / required VRAM 16GB+, another Qwen3.8-27B derivative model, is a generative model handling regular dialogue and text generation, this series is decisively different in its design, specializing in the “System One" API to return choices and probabilities directly without generating text.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 26.1B parameters (taken from the base model vllm-sr/d3)

Your VRAM Quantization File size Est. memory needed
24GB (RTX 4090 / 3090, etc.) Q4_K_M 18.1GB 21.7GB
32GB (RTX 5090, etc.) Q8_0 26.6GB 32.0GB
80GB class (A100 / H100) BF16 50.1GB 60.1GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the Q4_K_M build measures 5.96 bits per weight — about 37% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with 15–40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
ggml-org/Muse-Glimmer-30B-GGUF 29.8B 24GB apache-2.0 Muse-Glimmer-30B-GGUF 30B Multimodal Local Agent Model: 24GB+ VRAM (2026-11-17)
Cloudflare/clef-omni 35.3B 80GB apache-2.0 clef-omni 30B Multimodal Decision Model: 80GB+ VRAM (2026-11-17)
inclusionAI/ZwZ2-27B-GGUF 27.4B 24GB — ZwZ2-27B-GGUF Vision-Language Model: 24GB+ VRAM (2026-11-17)
ggml-org/OpenJev-GGUF 27.4B 24GB cc-by-nc-4.0 OpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM (2026-10-01)

How to Get It

The models released this time are distributed in GGUF format and can be obtained from their respective Hugging Face repositories. Since they are not gated models, they can be downloaded directly without applying for agreement to individual terms of use.

To run a model using llama.cpp’s server functionality, execute the following command:

llama serve -hf ggml-org/vllm-d3-GGUF

When running a smaller model, specify the repository name. For example, to run d3-flash:

llama serve -hf ggml-org/vllm-d3-flash-GGUF

After startup, you can send API requests via the /v1/systemone endpoint to execute decision-making processes. It also supports llama.app running in the browser.

Related Articles

What to Read Next

Sources