{"id":366,"date":"2026-09-09T18:13:44","date_gmt":"2026-09-09T09:13:44","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/09\/vllm-v0-29-0-released\/"},"modified":"2026-09-20T17:37:12","modified_gmt":"2026-09-20T08:37:12","slug":"vllm-v0-29-0-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/09\/vllm-v0-29-0-released\/","title":{"rendered":"vLLM v0.29.0 Released: Model Runner V2 Default and More"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nvllm-project\/vllm<\/a><\/td>\n<\/tr>\n
Version<\/td>\nv0.29.0<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-09-09<\/td>\n<\/tr>\n
License<\/td>\nApache-2.0<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code at collection time. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

vLLM v0.29.0 has been released. vLLM is a high-throughput and memory-efficient LLM inference and serving engine.<\/p>\n

The biggest change in this release is that Model Runner V2 (MRV2) is now the default for all models. This enables advanced optimizations to be available by default, such as CUDA graph memory profiling for automated KV cache sizing and batch-sharded sampling which reduces logits memory per step to 1\/TP.<\/p>\n

Breaking Changes and Deprecations<\/h2>\n

This update removes support for several model architectures and changes launch commands.<\/p>\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Old Architecture \/ Setting<\/th>\nNew State \/ Setting Method<\/th>\n<\/tr>\n<\/thead>\n
ArcticForCausalLM<\/code><\/td>\nRemoved (Snowflake Arctic)<\/td>\n<\/tr>\n
ChameleonForConditionalGeneration<\/code><\/td>\nRemoved (Meta Chameleon)<\/td>\n<\/tr>\n
Cheers<\/code> \/ CheersForConditionalGeneration<\/code><\/td>\nRemoved (ai9stars Cheers)<\/td>\n<\/tr>\n
Fairseq2LlamaForCausalLM<\/code><\/td>\nRemoved (fairseq2 Llama adapter)<\/td>\n<\/tr>\n
FireRedLIDForConditionalGeneration<\/code><\/td>\nRemoved (FireRedLID audio)<\/td>\n<\/tr>\n
GritLM<\/code><\/td>\nRemoved<\/td>\n<\/tr>\n
HCXVisionForCausalLM<\/code><\/td>\nRemoved (HyperCLOVAX-SEED-Vision V1)<\/td>\n<\/tr>\n
MPTForCausalLM<\/code> \/ MptForCausalLM<\/code><\/td>\nRemoved (MosaicML MPT)<\/td>\n<\/tr>\n
python -m vllm.entrypoints.openai.api_server<\/code><\/td>\nvllm serve<\/code><\/td>\n<\/tr>\n
VLLM_TEST_FORCE_FP8_MARLIN<\/code><\/td>\nUse linear_backend<\/code> or moe_backend<\/code><\/td>\n<\/tr>\n
VLLM_ROCM_USE_AITER_FP4_ASM_GEMM<\/code><\/td>\nRemoved<\/td>\n<\/tr>\n
PyAV (video decoder backend)<\/td>\nRemoved<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Users employing the above models or launching the API server using the legacy Python module format are required to migrate to the new architectures or switch to the new launch command.<\/p>\n

Key Changes<\/h2>\n

Model Runner V2 as Default and MRV1 Deprecation<\/h3>\n

Model Runner V2 (MRV2) is now the default for all models. MRV2 provides numerous optimization features including CUDA graph memory profiling for automatic KV cache sizing, batch-sharded sampling, prompt embeds, and speculation support for extract_hidden_states<\/code>.<\/p>\n

Accordingly, Model Runner V1 (MRV1) is deprecated and scheduled for removal in v0.32. However, if you are using features not yet supported by MRV2\u2014such as sequence parallelism, dual-batch overlap, elastic expert parallelism, custom logits processors, and certain speculative decoding methods\u2014it will automatically fall back to MRV1.<\/p>\n

FlashInfer All-Reduce Enabled by Default<\/h3>\n

The FlashInfer all-reduce backend is now enabled by default in CUDA groups using Tensor Parallel (TP). This is expected to improve communication efficiency in distributed environments. If you want to disable this behavior and use the legacy fallback chain, set the environment variable VLLM_ALLREDUCE_USE_FLASHINFER=0<\/code>. Note that if VLLM_BATCH_INVARIANT=1<\/code> is set, it is automatically excluded due to consistency requirements.<\/p>\n

Deterministic Behavior for Prefix Caching<\/h3>\n

Prefix-cache block hashes (NONE_HASH<\/code>) now operate deterministically by default. In previous versions, when PYTHONHASHSEED<\/code> was unset, different hashes were generated per process, requiring all nodes to fix the same PYTHONHASHSEED<\/code> when using distributed KV caches (such as the Mooncake connector). With this change, caches can be shared based on the default seed even across independent instances, reducing operational complexity.<\/p>\n

Introduction of New Admission Control Flags<\/h3>\n

To prevent increases in latency (TTFT) caused by request queueing, two new control flags have been added. --max-num-queued-reqs<\/code> places an upper limit on the total number of running and waiting requests. --max-num-queued-tokens<\/code> places an upper limit on the total prompt token numbers of requests currently in the prefill phase. This allows early rejection of requests when the prefill backlog threatens to exceed target latencies, helping to maintain Quality of Service (QoS).<\/p>\n

Supported Models and Hardware<\/h2>\n

New model additions include Tencent’s MoE model Hy4-preview (770B\/49B-active, with Gated DeepSeek Sparse Attention and native MTP), Qwen3.8-Flash-Next (BF16\/FP8\/NVFP4, with MTP), GraniteSWA, GraniteMoeSWA, NemotronH_Omni_Reasoning_V3 (with MTP), and Kimi K3 NVFP4 checkpoints. Additionally, model families such as FlexOlmo, Olmo3, and Hunyuan V1\/VL have been migrated from hand-in implementations to the Transformers modeling backend.<\/p>\n

For hardware and accelerators, DeepSeek V3.2 \/ GLM-5.2 DSA on NVIDIA environments is routed to optimized CUDA paths, while Blackwell benefits from auto-tuning E2E latency reduction and FA4 re-enabled. On AMD ROCm, W4A4 pre-shuffled asm GEMMs are now the default for improved throughput, alongside graph replay segfault fixes. On Intel XPU, the INC int4 W4A8 linear backend and AutoRound MXFP8 MoE have been added.<\/p>\n

How to Get It<\/h2>\n

For installing and updating vLLM, use the officially recommended uv<\/code> or pip<\/code>.<\/p>\n

uv pip install vllm\n<\/code><\/pre>\n
For PyPI (CUDA 13.0 environments), regular pip install vllm<\/code> is also available. Check the official release page for detailed platform-specific wheels and Docker image usage instructions.<\/p>\n
<\/p>\n
Releases Since Our Last Article<\/h2>\n
Compiled by Local Model Watch from the project’s GitHub releases: the versions between this release and the last one we covered, which did not get separate articles.<\/em> Full history: release tracker<\/a>.<\/em><\/p>\n
\n\n\n\n\n\n\n\n\n\n
Version<\/th>\nReleased<\/th>\nRelease notes<\/th>\n<\/tr>\n<\/thead>\n
v0.28.0<\/td>\n2026-08-26<\/td>\nGitHub<\/a><\/td>\n<\/tr>\n
v0.27.1<\/td>\n2026-08-11<\/td>\nGitHub<\/a><\/td>\n<\/tr>\n
v0.27.0<\/td>\n2026-08-11<\/td>\nGitHub<\/a><\/td>\n<\/tr>\n
v0.26.0<\/td>\n2026-07-27<\/td>\nGitHub<\/a><\/td>\n<\/tr>\n
v0.25.1<\/td>\n2026-07-14<\/td>\nGitHub<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n
<\/p>\n
Sources<\/h2>\n