Qwen3.8-27B-NVFP4-DGX-SPARK-V2 NVFP4 Quantized 27B Model: 24GB+ VRAM

At a Glance
| Item | Value |
|---|---|
| Repository | autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK-V2 |
| Published | 2026-10-28 |
| License | apache-2.0 |
| Formats | safetensors |
| Source type | Unverified (not confirmed by a primary source) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
This information has not been officially confirmed. It is reported by autotrust that a repository combining Qwen3.8-27B NVFP4 checkpoints with a 64k MTP (Multi-Token Prediction) recipe has been released.
This repository integrates the complete checkpoints of Qwen3.8-27B NVFP4 (including MTP tensors) with the “64k-vocabulary MTP recipe" provided by autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK. The 64k MTP optimization provided here is not written directly into the safetensors weights, but functions as a “runtime patch" loaded via an included vLLM plugin or an offline Python patch. This patch limits the draft head vocabulary size from 248,320 rows to 64,026 rows, and pads it to 64,064 rows for quantized GEMM kernels, significantly lightweighting the draft step. This mechanism is reported to make it possible to handle a larger number of speculative tokens. This configuration is designed targeting Blackwell GPUs (B200 and DGX Spark / GB10).
Specifications
- Architecture: Qwen3_5ForConditionalGeneration
- Quantization: NVFP4
- MTP: 64k-vocabulary MTP recipe
Performance
The model card does not include comparison tables with other models or benchmark figures.
However, according to the publisher’s report, applying this 64k MTP patch significantly reduces the size of the draft step, making it possible to handle a larger number of speculative tokens. As specific verification results, num_speculative_tokens: 7 for the B200 environment and 8 for the DGX Spark environment are reported as reasonable starting points.
This patch incorporates advanced quantization processing at runtime. Specifically, it collects draft heads from the target lm_head and performs quantization to Marlin NVFP4 W4A16 at runtime. Additionally, the five MTP linear projections are also quantized to Marlin NVFP4 at runtime. Note that this process is designed to have zero impact on the base target model itself and the verification lm_head, aiming to improve inference efficiency.
Strengths and Use Cases
This model is designed for high-speed inference on GPUs adopting the Blackwell architecture (B200 and DGX Spark / GB10). In particular, it features a configuration specialized for accelerating inference through speculative decoding using MTP (Multi-Token Prediction) technology.
Specifically, it has the following features:
- Efficient Speculative Decoding: Applying the 64k MTP patch significantly lightens the draft step and increases the number of speculative tokens that can be generated at once. Speculative token settings of about 7 tokens for B200 and 8 tokens for DGX Spark are recommended.
- Acceleration via Advanced Quantization: Improves inference efficiency by quantizing draft heads and MTP linear projections to Marlin NVFP4 at runtime.
- Support for Multimodal and Dialogue: According to tags, it supports text generation and conversational use cases, with support for English (en) and Chinese (zh).
Hardware Requirements
Estimated requirements (calculated by Local Model Watch)
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 24GB (RTX 4090 / 3090, etc.) | NVFP4 | 18.4GB | 22.1GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.
The publisher ships safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers or vLLM, using the memory figures in the table above.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
How to Get It
This model can be obtained from Hugging Face using the following steps. The distribution format is safetensors.
export MODEL_DIR="$HOME/models/Qwen3.8-27B-NVFP4-DGX-SPARK-V2"
HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK-V2 \
--local-dir "$MODEL_DIR"
cd "$MODEL_DIR"
Operation requires installing a dedicated vLLM plugin. Build the environment with the following steps:
python3 -m venv.venv
source.venv/bin/activate
python -m pip install -U pip
pip install "vllm==0.31.0"
pip install -e./mtp64k/plugin
Additionally, setting the following environment variables is recommended during execution:
export SPARK_DRAFT_HOME="$PWD/mtp64k/scripts"
export SPARK_DRAFT_VOCAB="$PWD/mtp64k/artifacts/draft_vocab_64k.pt"
export SPARK_DRAFT_HEAD_Q=marlin
export SPARK_DRAFT_MTP_Q=marlin
The supported inference engine is vLLM (version 0.31.0). SGLang and TensorRT-LLM are not supported.
Related Articles
- Qwen3.8-Flash-Next-E384-FP4-Local Vision-Language Model: ~96GB Memory
- TTS-V5 Chinese TTS Model: 4GB+ VRAM, File List
- TTS-V4 Chinese Text-to-Speech Model: 8GB+ VRAM, File List
- TTS-V3 Phone-Dialogue Specialized TTS Model: 8GB+ VRAM, File List
What to Read Next
- Find models by VRAM (This model runs from the 24GB tier) → Other models that run on a 24GB GPU
- Engines that run this model → vLLM
- What NVFP4 mean and where to get this model → NVFP4 / MXFP4 format guide and models / Quantization and model-format glossary
- Other models for the same task → Other text generation models
Sources
This article contains unverified information. We will append an update note once it is confirmed by a primary source.

