Qwen3.8-27B-NVFP4-DGX-SPARK-V2 NVFP4 Quantized 27B Model: 24GB+ VRAM

Qwen3.8-27B-NVFP4-DGX-SPARK-V2 NVFP4 Quantized 27B Model: 24GB+ VRAM

At a Glance

Item Value
Repository autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK-V2
Published 2026-10-28
License apache-2.0
Formats safetensors
Source type Unverified (not confirmed by a primary source)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

This information has not been officially confirmed. It is reported by autotrust that a repository combining Qwen3.8-27B NVFP4 checkpoints with a 64k MTP (Multi-Token Prediction) recipe has been released.

This repository integrates the complete checkpoints of Qwen3.8-27B NVFP4 (including MTP tensors) with the “64k-vocabulary MTP recipe" provided by autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK. The 64k MTP optimization provided here is not written directly into the safetensors weights, but functions as a “runtime patch" loaded via an included vLLM plugin or an offline Python patch. This patch limits the draft head vocabulary size from 248,320 rows to 64,026 rows, and pads it to 64,064 rows for quantized GEMM kernels, significantly lightweighting the draft step. This mechanism is reported to make it possible to handle a larger number of speculative tokens. This configuration is designed targeting Blackwell GPUs (B200 and DGX Spark / GB10).

Specifications

  • Architecture: Qwen3_5ForConditionalGeneration
  • Quantization: NVFP4
  • MTP: 64k-vocabulary MTP recipe

Performance

The model card does not include comparison tables with other models or benchmark figures.

However, according to the publisher’s report, applying this 64k MTP patch significantly reduces the size of the draft step, making it possible to handle a larger number of speculative tokens. As specific verification results, num_speculative_tokens: 7 for the B200 environment and 8 for the DGX Spark environment are reported as reasonable starting points.

This patch incorporates advanced quantization processing at runtime. Specifically, it collects draft heads from the target lm_head and performs quantization to Marlin NVFP4 W4A16 at runtime. Additionally, the five MTP linear projections are also quantized to Marlin NVFP4 at runtime. Note that this process is designed to have zero impact on the base target model itself and the verification lm_head, aiming to improve inference efficiency.

Strengths and Use Cases

This model is designed for high-speed inference on GPUs adopting the Blackwell architecture (B200 and DGX Spark / GB10). In particular, it features a configuration specialized for accelerating inference through speculative decoding using MTP (Multi-Token Prediction) technology.

Specifically, it has the following features:

  • Efficient Speculative Decoding: Applying the 64k MTP patch significantly lightens the draft step and increases the number of speculative tokens that can be generated at once. Speculative token settings of about 7 tokens for B200 and 8 tokens for DGX Spark are recommended.
  • Acceleration via Advanced Quantization: Improves inference efficiency by quantizing draft heads and MTP linear projections to Marlin NVFP4 at runtime.
  • Support for Multimodal and Dialogue: According to tags, it supports text generation and conversational use cases, with support for English (en) and Chinese (zh).

Hardware Requirements

Estimated requirements (calculated by Local Model Watch)

Your VRAM Quantization File size Est. memory needed
24GB (RTX 4090 / 3090, etc.) NVFP4 18.4GB 22.1GB

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.

The publisher ships safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers or vLLM, using the memory figures in the table above.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

How to Get It

This model can be obtained from Hugging Face using the following steps. The distribution format is safetensors.

export MODEL_DIR="$HOME/models/Qwen3.8-27B-NVFP4-DGX-SPARK-V2"

HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK-V2 \
  --local-dir "$MODEL_DIR"

cd "$MODEL_DIR"

Operation requires installing a dedicated vLLM plugin. Build the environment with the following steps:

python3 -m venv.venv
source.venv/bin/activate
python -m pip install -U pip

pip install "vllm==0.31.0"
pip install -e./mtp64k/plugin

Additionally, setting the following environment variables is recommended during execution:

export SPARK_DRAFT_HOME="$PWD/mtp64k/scripts"
export SPARK_DRAFT_VOCAB="$PWD/mtp64k/artifacts/draft_vocab_64k.pt"
export SPARK_DRAFT_HEAD_Q=marlin
export SPARK_DRAFT_MTP_Q=marlin

The supported inference engine is vLLM (version 0.31.0). SGLang and TensorRT-LLM are not supported.

Related Articles

What to Read Next

Sources

This article contains unverified information. We will append an update note once it is confirmed by a primary source.