Qwen3.8-27B-DSpine-b16 Draft Model for Speculative Decoding: 8GB+ VRAM

At a Glance
| Item | Value |
|---|---|
| Repository | tencent/Qwen3.8-27B-DSpine-b16 |
| Family guide | Qwen3.8 guide (10 articles) |
| Publisher guide | Tencent: models and licenses |
| Published | 2026-11-07 |
| License | apache-2.0 |
| Formats | safetensors |
| Paper | arXiv:2609.36173, arXiv:2607.05147 |
| Source type | Unverified (not confirmed by a primary source) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
This information has not been confirmed as an official announcement and is based on unverified information published in a repository on Hugging Face.
It is reported that Tencent has released a draft model for speculative decoding named “tencent/Qwen3.8-27B-DSpine-b16" for the large-scale model “Qwen3.8-27B". This model is said to be a checkpoint adopting the “DSpine" method proposed in the paper “Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding" (arXiv:2609.36173). While generating candidate tokens in parallel, it passes the preceding hidden states predicted at each layer to adjacent positions to reflect causal relationships along the depth direction of the network, aiming to improve the acceptance length and throughput in speculative decoding.
Specifications
- Architecture: DFlashDraftModel
- Number of layers: 5 layers
- Block size: 16
- Sliding window: 8,192 tokens
- Base model: Qwen/Qwen3.8-27B
Performance
According to the measurement results published in the model card by the releasing party, comparative verification was conducted using SGLang (version 0.5.22, BF16, FlashAttention 3, tensor parallelism 8) as the inference engine against the existing method DFlash2 (block size 8) and standard autoregressive generation (AR).
Comparisons of the average acceptance length on general benchmarks are as follows:
→ Scroll horizontally to see all columns
| Task | Greedy DFlash2 | Greedy DSpine | Sampled DFlash2 | Sampled DSpine |
|---|---|---|---|---|
| GSM8K | 5.69 | 7.20 | 5.45 | 6.31 |
| MATH-500 | 5.49 | 6.49 | 5.27 | 5.66 |
| HumanEval | 4.87 | 5.71 | 4.34 | 4.71 |
| MBPP | 5.04 | 6.14 | 4.79 | 5.41 |
| MT-Bench | 4.11 | 4.48 | 3.89 | 4.03 |
Additionally, the acceptance length in agent tasks is reported as follows:
→ Scroll horizontally to see all columns
| Benchmark | low / DFlash2 | low / DSpine | xhigh / DFlash2 | xhigh / DSpine |
|---|---|---|---|---|
| WorkBuddy Code | 4.19 | 5.08 | 4.06 | 4.92 |
| WorkBuddy Web | 3.95 | 4.91 | 3.95 | 5.01 |
| WorkBuddy Office | 4.24 | 5.63 | 4.20 | 5.49 |
| SWE-bench Pro | 4.19 | 4.99 | 4.00 | 4.77 |
| τ²-bench | 4.70 | 5.53 | 4.29 | 4.95 |
Furthermore, the measurement results for end-to-end output throughput (tokens/s) and speedup factor relative to autoregressive generation (×AR) in agent environments are as follows:
→ Scroll horizontally to see all columns
| Benchmark | low / AR | low / DFlash2 (×AR) | low / DSpine (×AR) | xhigh / AR | xhigh / DFlash2 (×AR) | xhigh / DSpine (×AR) |
|---|---|---|---|---|---|---|
| WorkBuddy Code | 253.35 | 590.27 (2.33×) | 686.23 (2.71×) | 193.00 | 550.40 (2.85×) | 613.55 (3.18×) |
| WorkBuddy Web | 247.40 | 639.74 (2.59×) | 773.19 (3.13×) | 252.26 | 734.04 (2.91×) | 840.14 (3.33×) |
| WorkBuddy Office | 210.03 | 664.77 (3.17×) | 805.53 (3.84×) | 215.05 | 662.34 (3.08×) | 763.83 (3.55×) |
| SWE-bench Pro | 241.86 | 657.71 (2.72×) | 713.41 (2.95×) | 248.17 | 711.60 (2.87×) | 791.30 (3.19×) |
| τ²-bench | 301.39 | 891.37 (2.96×) | 964.30 (3.20×) | 340.39 | 871.98 (2.56×) | 978.39 (2.87×) |
From these measurement results, it can be seen that DSpine consistently records higher acceptance lengths than the comparison target DFlash2 across all evaluated tasks and sampling settings. In particular, it reaches 7.20 tokens under the Greedy setting for the arithmetic reasoning task GSM8K and 5.63 tokens under the low setting for WorkBuddy Office, demonstrating a significant extension in acceptance length compared to prior methods. On the other hand, for chat tasks like MT-Bench, the increase in acceptance length is relatively small, confirming that the acceleration effects obtained vary depending on the nature of the task. In terms of throughput, it achieves up to 3.84× the speed of standard autoregressive generation (low setting of WorkBuddy Office), showing improvements in generation efficiency that surpass DFlash2.
Note that this model is solely responsible for draft generation, and the accuracy of the output text and the reasoning capabilities themselves depend on the base model, Qwen3.8-27B. For reference, measurements published by the original model creators for Qwen3.8-27B report performances such as 73.0 on Terminal Bench 2.1, 61.7 on SWE-bench Pro, and 89.2 on GPQA Diamond.
Strengths and Use Cases
This model is a dedicated draft model for speculative decoding designed to operate in combination with the base model “Qwen3.8-27B". It is not intended to complete text generation on its own, but rather its main use case is to accelerate the token generation processing of the target model side.
According to the description in the model card and verification results, utilization is anticipated in the following use cases and environments:
- Acceleration of agent environments and tool-calling tasks: High throughput improvements can be expected in agent tasks involving long contexts or frequent feedback from tool execution results. In the publisher’s verification, a speedup of approximately 2.7× to 3.8× compared to autoregressive generation was reported in environments such as WorkBuddy and SWE-bench Pro.
- Improvement of reasoning efficiency for mathematical reasoning and coding: In tasks with long reasoning steps such as GSM8K and HumanEval, it is said to achieve longer acceptance lengths than the conventional draft method (DFlash2). This is expected to have the effect of shortening waiting times for programming assistance and logical reasoning processes that involve a high number of trial steps and large volumes of token generation.
- High-throughput inference serving using SGLang: This model is optimized for operation on SGLang, making it suitable for configurations that enhance overall processing capacity while suppressing generation latency in API server environments handling batch processing and parallel requests.
Note that the tokenizer, embedding layer, and final LM head utilize those of the target model, Qwen3.8-27B. Therefore, characteristics such as output language capability, knowledge, and supported languages basically conform to the base model’s specifications.
How It Differs from Similar Models
As derivative models also based on Qwen3.8-27B, several projects have been released in the past.
The “clef" structured decision-making model: requiring 12GB+ VRAM with GGUF versions available, previously introduced on our site, is a specialized model that takes states like text, images, and videos along with defined question schemas as input and directly outputs decisions in a single forward pass. In contrast, the current DSpine does not change the model’s generation paradigm, differing significantly in that it is a draft model that externally accelerates the normal generation process of Qwen3.8-27B through speculative decoding.
Additionally, the “OrcaSAQ-2-27B" text generation model: response examples on our site & requiring 16GB+ VRAM is a derivative version aimed at reducing the memory required for operation by compressing the base model itself to low bits using proprietary sensitivity-aware mixed-precision quantization. In contrast to this, DSpine adopts a speed-oriented approach that boosts output throughput by running inference in parallel with the target model, rather than saving resources through model compression.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 2.0B parameters
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 8GB (RTX 4060 / 3060 Ti, etc.) | BF16 | 3.8GB | 4.5GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Recent Models in the Same Size Class
Models with up to 4B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf | 389M | 4GB | apache-2.0 | MiniCPM-V-4.7-35B-A3B-DSpark-gguf Draft Model for Speedup: 4GB+ VRAM (2026-11-17) |
| openbmb/MiniCPM-V-4.7-1B-DSpark-gguf | 214M | 4GB | apache-2.0 | MiniCPM-V-4.7-1B-DSpark-gguf Text Generation Model: 4GB+ VRAM (2026-11-17) |
| IFM/Jais-3-0.9B-GGUF | 1.1B | 4GB | apache-2.0 | Jais-3-0.9B-GGUF Bilingual Dense Model: 4GB+ VRAM (2026-11-17) |
| nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 | 4.0B | 4GB | openmdw-1.1 | NVIDIA Releases Dory1.0-4B: A Recurrent 4B LLM with Dynamic Loops (2026-11-17) |
| LiquidAI/LFM2.5-24B-A2B-DSpark | 328M | 4GB | lfm1.0 | LFM2.5-24B-A2B-DSpark Speculative Decoding Draft Model: 4GB+ VRAM (2026-11-17) |
How to Get It
The weights of this model are distributed on Hugging Face, and the Apache-2.0 license applies. Since access restrictions (Gated) are not set, direct downloading is possible without individual agreement to the terms of use.
The distribution format is provided as safetensors. SGLang is envisioned as the inference engine, and the package installation from the official repository and startup specifying dedicated options are guided.
Tool installation can be executed with the following command:
git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e.
A command example for starting an inference server linking with the target model (Qwen/Qwen3.8-27B) using SGLang is as follows:
sglang serve \
--model-path Qwen/Qwen3.8-27B \
--speculative-algorithm DSPINE \
--speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \
--speculative-num-draft-tokens 16 \
--dtype bfloat16 \
--attention-backend fa3 \
--speculative-draft-attention-backend fa3 \
--linear-attn-backend triton \
--enable-linear-replayssm-spec \
--mamba-ssm-dtype float32 \
--mamba-radix-cache-strategy extra_buffer \
--max-mamba-cache-size 64 \
--mem-fraction-static 0.80 \
--context-length 131072 \
--max-running-requests 8 \
--cuda-graph-max-bs-decode 8 \
--min-free-slots-delay 1 \
--sampling-backend flashinfer \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--trust-remote-code
For detailed execution procedures such as runtime settings and dynamic verification tuning methods, please refer to the publisher’s repository and guide documents.
Related Articles
- Qwen3.8-27B Vision-Language Model: Our Test Answers, 8GB+ VRAM
- Ternary-Bonsai-2-27B-gguf: Our Test Answers, 8GB+ VRAM
- clef Structured Decision-Making Model: 12GB+ VRAM, GGUF Builds
- vllm-d3 GGUF Models Released: Decision Models for Local Run
What to Read Next
- Find models by VRAM (This model runs from the 8GB tier) → Other models that run on a 8GB GPU
- Explore the same model family → Qwen3.8 family overview (10 articles, 17 converted builds)
- Formats this model is available in → Safetensors format guide and models
- Learn about the publisher → Tencent: models, licenses and articles
- Other models for the same task → Other text generation models
Sources
- tencent/Qwen3.8-27B-DSpine-b16 (Hugging Face)
- Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding (arXiv:2609.36173)
- Qwen/Qwen3.8-27B (Hugging Face)
This article contains unverified information. We will append an update note once it is confirmed by a primary source.

