vLLM v0.32.0 Released: Major Updates and Enhancements

vLLM v0.32.0 Released: Major Updates and Enhancements

At a Glance

Item Value
Repository vllm-project/vllm
Version v0.32.0
Published 2026-10-18
License Apache-2.0
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

vLLM version 0.32.0 has been released. vLLM is an open-source inference and serving engine designed for high-throughput and memory-efficient LLM inference.

This release introduces breaking changes related to the operational base, such as dropping support for Python 3.10 and tightening default CORS settings, alongside significantly expanded environment support for low-precision KV caches including NVFP4 and 4-bit formats. Furthermore, multiple security vulnerabilities have been patched, and various optimizations have been introduced to boost startup speed and throughput.

Breaking Changes and Deprecations

This release includes several breaking changes that affect the runtime environment and API behavior.

Item Before (Old) After (New)
Supported Python versions Python 3.10 and later Python 3.11–3.14 (Python 3.10 support dropped)
Default CORS settings Allow all origins (["*"]) Denied by default (empty list, explicit --allowed-origins required)
Minimum dependency versions transformers>=5.10.4 transformers>=5.16.1, tokenizers>=0.23.2
max_tokens memory reservation behavior Pre-reserved slots based on context length Treated as an upper bound, pre-reservation removed
Removed features and paths mamba_cache_mode="all", cascade attention, torch.compile SP/AsyncTP paths Completely removed
Responses API stream events Previous reasoning event names Changed to new stream event names

These changes impact users running Python 3.10 environments or production setups where vLLM servers are directly accessed from browsers via web frontends. In particular, due to the CORS specification change, systems integrating with browsers must explicitly specify --allowed-origins upon startup.

Key Changes

Critical Security Vulnerability Fixes and Additional Defenses

A DoS vulnerability (GHSA-4hhp-h66f-j5j7) causing CPU and memory exhaustion via chat template rendering has been fixed, introducing static AST validation and timeout configuration (VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT). Additionally, countermeasures have been added for a bug where off-by-one errors in the tokenizer’s max_token_id caused CUDA assert failures (GHSA-q97c-p368-75c4), and a crash vulnerability caused by P2P ZMQ socket exhaustion (GHSA-frvg-qw88-hww9), introducing connection limits (VLLM_P2P_MAX_PEERS, default 128). Robustness has also been enhanced in multiple areas, such as restricting untrusted image loading to raster formats via an allowlist, HMAC signature verification for shared memory cache handles, and rejecting nested structured JSON schemas exceeding 64 levels (returning HTTP 400).

Expanded Support for Low-Precision KV Caches

The usage scope for low-precision KV caches, which improve memory efficiency, has been expanded. NVFP4 KV cache now supports NVIDIA SM8x, SM90, and SM12x, and is available for XQA decoding on SM12x and MiniMax-M3 sparse attention on SM100. Additionally, FP8 KV cache becomes the default for GLM-5.x in SM100 environments, improving throughput by 2.3–5.5%, while a 4-bit format ultraquant_4bit KV cache—offering roughly double the density of FP8—has been introduced for ROCm environments.

Model Runner V2 (MRV2) Enhancements and Default Adoption on ROCm

Model Runner V2, the next-generation runner, adds support for variable-length drafts in GPU n-gram speculative decoding, Elastic EP, combined pipeline parallelism and context parallelism for MLA models, and standard torch.compile mode. Furthermore, MRV2 is now set as the default runner for all models in ROCm environments.

Faster Startup and Improved Serving Performance

The new vllm download-kernels command allows users to install pre-compiled FlashInfer kernels, reducing cold-start times. Other additions include --preload-autotune to retain autotune tables across restarts, tensor arenas (--enable-shm-tensor-arena, enabled by default) to transfer multimodal tensors to workers via shared memory in a single copy, and loading tokenizers from local snapshots. Furthermore, in environments where --api-server-count is set to 2 or more, connections are now routed to the API worker with the lowest load, increasing requests per second by 43–85% for short requests.

Supported Models and Hardware

Newly Added Models and Expanded Architectures

This version adds support for several new model architectures.
– EmbeddingGemma 2 (google/embeddinggemma-2): Supports omni-modal embedding models for text, image, audio, and video. By specifying --limit-mm-per-prompt, encoders for modalities not used within the prompt can be skipped for efficient execution.
– MiniCPM-V 4.7: Newly supported for serving.
– Nemotron 3.5 ASR: Supported as a streaming speech-to-text model.
– Structured Decision-Making Models: Supports execution of Cloudflare/clef-omni, LiquidAI/d1-3B, and DiffusionGemma via the newly established /v1/systemone endpoint.
– Others: Added support for PLaMo 3 inference and tool parsers, as well as the zerank-2 reranker score template.

In addition, video input is now supported in the Transformers modeling backend, and models such as Glm, Arcee, CWM, Mellum, GPT-NeoX, Phi, Seed-OSS, and Jais2 can now be served via this backend.

Inference Optimizations and New Features for Existing Models

A number of architecture-specific optimizations have been introduced for major large-scale models.
– DeepSeek-V4 and V4.1: Features bounded replays for decoder-side SWA (Sliding Window Attention) and optimized replay layers within CUDA graphs. It introduces packed FP8 Engram exchanges across data parallelism (DP), faster Engram searches during CPU offloading, and drafting fusing DeepSeek-V4 with MTP (Multi-Token Prediction). Furthermore, encoder CUDA graph packing for V4.1 and Kimi-K3 reduces average TTFT (Time To First Token) by 11%.
– GLM-5.3 and GLM-5.3-Flash: Reusing sparse MLA (Multi-head Latent Attention) index transformations across layers sped up the corresponding kernel by 3.5–3.9x, and fused multi-step draft decoding was added. Support for fp8_ds_mla on SM90, a 4.4–7.7% TTFT improvement via skipped qlnorm for MHA, and indexer prefill distribution via tensor parallelism at 1-million context (reducing TTFT by up to 25%) are also included.
– Qwen Family: Qwen3.8-Flash-Next (Qwen4Exp) integrates QSA projections into a single GEMM, achieving a 1.45–1.68x speedup. Sequence parallelism support via --enable-hc-sp and low-latency decoding GEMM plans for SM100 (up to 2.7x faster than cuBLAS for small M sizes) are also provided. Additionally, M-RoPE cache sizes for Qwen3-VL, Qwen3.5, and Qwen3.8 have been reduced to a quarter.
– Kimi-K3 and MiniMax-M3: FlashInfer fused speculative KDA on SM100 and a fused vision processor improving preprocessing speed by 10–20x at 1080p–8K resolutions have been added for Kimi-K3. For MiniMax-M3, MSA prefill combining FP8 KV cache and E4M3 queries is 26–37% faster.
– General Multimodal: Device-side input normalization was introduced for Qwen3-VL, Qwen3.5, Qwen4Exp, GLM-V, GLM-OCR, Llama Nemotron VL, and Kimi K2.5/K3, sorting encoder inputs by modality to boost request processing capacity by 6.5–8.2%.

Hardware Platform Enhancements

Significant performance enhancements have been made not only for NVIDIA GPUs, but also for AMD ROCm, Intel XPU, and CPU environments.
– NVIDIA: SM100 now supports native 64-head FlashMLA sparse prefill, improving TTFT by 1.16–1.18x. Split-K attention for SM120 and OAI Triton MXFP4 MoE kernels for SM12x have also been added.
– AMD ROCm: AITER MegaMoEV2 via --moe-backend aiter_mega_moe was introduced for DeepSeek-V4, with support for shared expert combination (3–8% improvement). Mono decode (VLLM_ROCM_MONO_DECODE=1) for DeepSeek-V4.1-Flash on MI355X, Gluon sparse MLA for GLM-5.2 (up to 16% improvement), ROCM_SEGMENTED_ATTN for RDNA, and W4A16 tuning on gfx11 (up to 23% TTFT improvement) are also available.
– Intel XPU: MLA support in FLASH_ATTN_MLA, FP8 GEMMs (2.0–2.73x faster) and fused MoE configurations for Intel Arc Pro B70, and fused SYCL LayerNorm have been introduced.
– CPU: Vectorized samplers for s390x environments (up to 58x), causal conv1d optimizations for aarch64 (approx. 9% improvement), and AWQ/GPTQ support for POWER10 have been added.

Quantization Feature Expansion

On the quantization front, online requantization of linear layers (--quantization-config.linear fp8_per_channel) is now possible, alongside token-wise NVFP4 CuTe-DSL MoE and MXFP4 MoE oracles equipped with CUTLASS W4A4 fallbacks. ModelOpt’s IQ2_XS, IQ2_XXS, and Q8_0 formats are also supported for SM120/SM121.

How to Get It

When updating to vLLM v0.32.0, ensure that the Python version in your execution environment is 3.11–3.14 beforehand.

Update packages using the recommended uv or standard pip commands. Running vllm download-kernels after installation deploys pre-compiled kernels and reduces startup time.

uv pip install vllm
vllm download-kernels

If using standard pip, execute the following:

pip install vllm
vllm download-kernels

For ROCm and Intel XPU environments, install by specifying their respective dedicated extra index URLs.

# For AMD ROCm environments
pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.32.0/rocm723

# For Intel XPU environments
uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.32.0/xpu --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match

When using Docker containers, pull and update to the official Docker images.

# CUDA 13.0 environment (default)
docker pull vllm/vllm-openai:v0.32.0

# CUDA 12.9 environment
docker pull vllm/vllm-openai:v0.32.0-cu129

# AMD ROCm environment
docker pull vllm/vllm-openai-rocm:v0.32.0

# CPU environment
docker pull vllm/vllm-openai-cpu:v0.32.0

# Intel XPU environment
docker pull vllm/vllm-openai-xpu:v0.32.0

Related Articles

What to Read Next

Sources