MiniCPM-V-4.7-35B-A3B-DSpark-gguf Draft Model for Speedup: 4GB+ VRAM

MiniCPM-V-4.7-35B-A3B-DSpark-gguf Draft Model for Speedup: 4GB+ VRAM

At a Glance

Item Value
Repository openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf
Family guide MiniCPM-V-4.7-DSpark guide (2 articles)
Publisher guide OpenBMB: models and licenses
Published 2026-10-15
License apache-2.0
Formats GGUF
Paper arXiv:2607.05147
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

OpenBMB has released a GGUF (llama.cpp) quantized version of MiniCPM-V-4.7-35B-A3B-DSpark. This is not a standalone generative model, but a “speculative decoding" draft model designed to accelerate inference for the vision-language model MiniCPM-V-4.7-35B-A3B. When running MiniCPM-V-4.7-35B-A3B in llama.cpp or vLLM, simply adding this DSpark model is expected to improve response speeds.

According to the abstract of the paper “DSpark" referenced by the model card, DSpark is a speculative decoding method that combines the high throughput of parallel draft generation with adaptive verification based on load conditions. It is described as optimizing serving efficiency by maintaining dependency between tokens via a semi-autoregressive architecture to prevent draft quality degradation, while dynamically adjusting verification lengths according to confidence scores.

Specifications

  • Architecture: Qwen3DSparkModel (qwen3 family)
  • Draft parameter count: 389,223,169 (approx. 390M)
  • Draft layers: 5
  • Draft tokens per forward pass: 15
  • Target model hidden state reference layers: [3, 11, 19, 27, 35]
  • Checkpoint precision: BF16
  • Paired target model: MiniCPM-V-4.7-35B-A3B (no specific dense/MoE designation mentioned for the base model)

Performance

1

Domain Accept Length
Vision
MMMU 5.098
MathVista 6.126
CharXiv_reasoning_val 5.479
OCRBench 5.407
Text
Math 6.687
Code 7.387
General 3.501

For text domains, Code is the highest across all categories at 7.387, suggesting that tasks with predictable patterns like code generation yield higher draft acceptance rates. On the other hand, General is the lowest at 3.501, indicating that acceptance lengths are harder to extend in free-form general response generation. In the vision domain, MathVista, which measures mathematical problems involving figures and graphs in images, scored the highest at 6.126, while MMMU, which tests college-level image understanding, resulted in a relatively low 5.098. OCRBench, which measures text recognition in images, stands at 5.407, placing it in the middle among the four vision categories. Overall, the table shows a trend where inputs with easily determinable structures, such as code and mathematical formulas, yield larger speedup effects, whereas free-form general dialogue sees somewhat more modest gains.

Strengths and Use Cases

This model is a dedicated speculative decoding draft model designed to be combined with the multimodal model MiniCPM-V-4.7-35B-A3B to accelerate inference. Rather than acting as a standalone generative model for independent conversation or image recognition, it plays the role of pre-reading and proposing token sequences generated by the target model. According to the model card, quantization of the draft model has absolutely no effect on the content of the final generated output itself, influencing only the accept length (the number of tokens accepted per step).

The target model and this draft model are tagged with vision-understanding, tool-calling, conversational, en, zh, on-device, and edge-ai. From this, the main use cases are accelerating English and Chinese text generation, conversation, tool calling, and general vision tasks accompanied by image inputs.

As shown in the aforementioned accept length data, it achieves high accept lengths particularly in tasks requiring structured outputs such as code generation and mathematical reasoning, and is greatly expected to improve speeds in programming assistance and calculation/logical thinking tasks. Furthermore, in image understanding tasks, stable accept lengths have been reported in MathVista (which solves graphs and geometric problems) and OCRBench (text recognition), making it well-suited for accelerating workflows involving visual reasoning and document image reading. The model is designed with edge environments and on-device deployment in mind, making it suitable for environments where response performance needs to be enhanced under limited computational resources.

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 389M parameters (taken from the base model openbmb/MiniCPM-V-4.7-35B-A3B-DSpark)

Your VRAM Quantization File size Est. memory needed
4GB (laptop iGPU / phone class) F16 0.7GB 0.9GB

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Runs in Ollama, LM Studio and llama.cpp as-is.

It is distributed in GGUF, so no conversion is needed.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Recent Models in the Same Size Class

Models with up to 4B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

→ Scroll horizontally to see all columns

Model Parameters Smallest VRAM tier License Our article
openbmb/MiniCPM-V-4.7-1B-DSpark-gguf 214M 4GB apache-2.0 MiniCPM-V-4.7-1B-DSpark-gguf Text Generation Model: 4GB+ VRAM (2026-11-17)
IFM/Jais-3-0.9B-GGUF 1.1B 4GB apache-2.0 Jais-3-0.9B-GGUF Bilingual Dense Model: 4GB+ VRAM (2026-11-17)
nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 4.0B 4GB openmdw-1.1 NVIDIA Releases Dory1.0-4B: A Recurrent 4B LLM with Dynamic Loops (2026-11-17)
LiquidAI/LFM2.5-24B-A2B-DSpark 328M 4GB lfm1.0 LFM2.5-24B-A2B-DSpark Speculative Decoding Draft Model: 4GB+ VRAM (2026-11-17)
LiquidAI/LFM2.5-24B-A2B-DSpark-GGUF 328M 4GB lfm1.0 LFM2.5-24B-A2B-DSpark-GGUF Text Generation Model: 4GB+ VRAM (2026-11-17)

How to Get It

This model is distributed in GGUF format and can be obtained from the Hugging Face repository. Since it is not a gated model, it can be downloaded directly without prior application or consent procedures. Supported inference engines include llama.cpp and vLLM.

When using it with llama.cpp, download the GGUF file of the target model itself, the vision projector (mmproj), and the GGUF file of this DSpark draft model. The model card provides the following download example using huggingface-cli:

huggingface-cli download openbmb/MiniCPM-V-4.7-35B-A3B-gguf MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf mmproj-model-f16.gguf --local-dir./35B-A3B-gguf
huggingface-cli download openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf dspark-MiniCPM-V-4_7-35B-A3B-Q8_0.gguf --local-dir./35B-A3B-gguf

Using the model requires a build of llama.cpp that supports the dflash architecture and the DSpark Markov head. When running llama-cli, specify the draft model with the -md option and enable speculative decoding by specifying --spec-type draft-dspark. An example command to run it with an image and prompt via the CLI tool is as follows:

llama-cli \
  -m./35B-A3B-gguf/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
  --mmproj./35B-A3B-gguf/mmproj-model-f16.gguf \
  -md./35B-A3B-gguf/dspark-MiniCPM-V-4_7-35B-A3B-Q8_0.gguf \
  --spec-type draft-dspark \
  --image image.jpg -p "Describe this image." \
  -ngl 99 -ngld 99 -fa on -c 8192

When launching as a server, configure llama-server by passing parameters such as --spec-draft-n-max 15:

llama-server \
  -m./35B-A3B-gguf/MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
  --mmproj./35B-A3B-gguf/mmproj-model-f16.gguf \
  -md./35B-A3B-gguf/dspark-MiniCPM-V-4_7-35B-A3B-Q8_0.gguf \
  --spec-type draft-dspark --spec-draft-n-max 15 \
  -ngl 99 -ngld 99 -fa on -c 8192 --jinja --port 8080

Additionally, when serving with vLLM, a configuration is provided to incorporate this DSpark model via --speculative-config targeting the original BF16 checkpoint:

vllm serve \
  --model openbmb/MiniCPM-V-4.7-35B-A3B \
  --served-model-name MiniCPM-V-4.7-35B-A3B \
  --tensor-parallel-size 2 \
  --max-model-len 40960 \
  --max-num-batched-tokens 40960 \
  --max-num-seqs 1 \
  --attention-backend FLASH_ATTN \
  --mm-encoder-attn-backend FLASH_ATTN \
  --mm-processor-kwargs '{"downsample_mode":"4x"}' \
  --trust-request-mm-kwargs \
  --disable-custom-all-reduce \
  --compilation-config '{"pass_config":{"fuse_allreduce_rms":false}}' \
  --speculative-config '{"method":"dspark","model":"openbmb/MiniCPM-V-4.7-35B-A3B-DSpark","num_speculative_tokens":15,"enable_adaptive_verification":false}' \
  --default-chat-template-kwargs '{"enable_thinking":false}'

The model card mentions that for the draft’s quantization type, one should choose the smallest one that fits into the environment, with Q8_0 and Q6_K serving as good default choices.

Related Articles

What to Read Next

Sources