vLLM v0.29.0 Released: Model Runner V2 Default and More

September 9, 2026September 20, 2026

At a Glance

Values determined by this site’s code at collection time. Dates are JST.

Overview

vLLM v0.29.0 has been released. vLLM is a high-throughput and memory-efficient LLM inference and serving engine.

The biggest change in this release is that Model Runner V2 (MRV2) is now the default for all models. This enables advanced optimizations to be available by default, such as CUDA graph memory profiling for automated KV cache sizing and batch-sharded sampling which reduces logits memory per step to 1/TP.

Breaking Changes and Deprecations

This update removes support for several model architectures and changes launch commands.

Users employing the above models or launching the API server using the legacy Python module format are required to migrate to the new architectures or switch to the new launch command.

Key Changes

Model Runner V2 as Default and MRV1 Deprecation

Model Runner V2 (MRV2) is now the default for all models. MRV2 provides numerous optimization features including CUDA graph memory profiling for automatic KV cache sizing, batch-sharded sampling, prompt embeds, and speculation support for extract_hidden_states.

Accordingly, Model Runner V1 (MRV1) is deprecated and scheduled for removal in v0.32. However, if you are using features not yet supported by MRV2—such as sequence parallelism, dual-batch overlap, elastic expert parallelism, custom logits processors, and certain speculative decoding methods—it will automatically fall back to MRV1.

FlashInfer All-Reduce Enabled by Default

The FlashInfer all-reduce backend is now enabled by default in CUDA groups using Tensor Parallel (TP). This is expected to improve communication efficiency in distributed environments. If you want to disable this behavior and use the legacy fallback chain, set the environment variable VLLM_ALLREDUCE_USE_FLASHINFER=0. Note that if VLLM_BATCH_INVARIANT=1 is set, it is automatically excluded due to consistency requirements.

Deterministic Behavior for Prefix Caching

Prefix-cache block hashes (NONE_HASH) now operate deterministically by default. In previous versions, when PYTHONHASHSEED was unset, different hashes were generated per process, requiring all nodes to fix the same PYTHONHASHSEED when using distributed KV caches (such as the Mooncake connector). With this change, caches can be shared based on the default seed even across independent instances, reducing operational complexity.

Introduction of New Admission Control Flags

To prevent increases in latency (TTFT) caused by request queueing, two new control flags have been added. --max-num-queued-reqs places an upper limit on the total number of running and waiting requests. --max-num-queued-tokens places an upper limit on the total prompt token numbers of requests currently in the prefill phase. This allows early rejection of requests when the prefill backlog threatens to exceed target latencies, helping to maintain Quality of Service (QoS).

Supported Models and Hardware

New model additions include Tencent’s MoE model Hy4-preview (770B/49B-active, with Gated DeepSeek Sparse Attention and native MTP), Qwen3.8-Flash-Next (BF16/FP8/NVFP4, with MTP), GraniteSWA, GraniteMoeSWA, NemotronH_Omni_Reasoning_V3 (with MTP), and Kimi K3 NVFP4 checkpoints. Additionally, model families such as FlexOlmo, Olmo3, and Hunyuan V1/VL have been migrated from hand-in implementations to the Transformers modeling backend.

For hardware and accelerators, DeepSeek V3.2 / GLM-5.2 DSA on NVIDIA environments is routed to optimized CUDA paths, while Blackwell benefits from auto-tuning E2E latency reduction and FA4 re-enabled. On AMD ROCm, W4A4 pre-shuffled asm GEMMs are now the default for improved throughput, alongside graph replay segfault fixes. On Intel XPU, the INC int4 W4A8 linear backend and AutoRound MXFP8 MoE have been added.

How to Get It

For installing and updating vLLM, use the officially recommended uv or pip.

uv pip install vllm

For PyPI (CUDA 13.0 environments), regular pip install vllm is also available. Check the official release page for detailed platform-specific wheels and Docker image usage instructions.

Releases Since Our Last Article

Compiled by Local Model Watch from the project’s GitHub releases: the versions between this release and the last one we covered, which did not get separate articles. Full history: release tracker.

Sources

• vllm-project/vllm v0.29.0 Release Notes
• vllm-project/vllm PR #52998
• vllm-project/vllm PR #51875
• vllm-project/vllm PR #49445
• vllm-project/vllm PR #53608
• vllm-project/vllm PR #53615
• vllm-project/vllm PR #54231
• vllm-project/vllm PR #52131
• vllm-project/vllm PR #52182

Update History

• 2026-09-20: Rewrote the article from re-collected sources and restored it from draft to published.