{"id":12416,"date":"2026-11-17T11:01:21","date_gmt":"2026-11-17T02:01:21","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=12416"},"modified":"2026-11-17T11:10:26","modified_gmt":"2026-11-17T02:10:26","slug":"vllm-v0320-released-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/vllm-v0320-released-2\/","title":{"rendered":"vLLM v0.32.0 Released: Major Updates and Enhancements"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/vllm-project\/vllm\">vllm-project\/vllm<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/vllm-project\/vllm\/releases\/tag\/v0.32.0\">v0.32.0<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-18<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>Apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>vLLM version 0.32.0 has been released. vLLM is an open-source inference and serving engine designed for high-throughput and memory-efficient LLM inference.<\/p>\n<p>This release introduces breaking changes related to the operational base, such as dropping support for Python 3.10 and tightening default CORS settings, alongside significantly expanded environment support for low-precision KV caches including NVFP4 and 4-bit formats. Furthermore, multiple security vulnerabilities have been patched, and various optimizations have been introduced to boost startup speed and throughput.<\/p>\n<h2>Breaking Changes and Deprecations<\/h2>\n<p>This release includes several breaking changes that affect the runtime environment and API behavior.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Before (Old)<\/th>\n<th>After (New)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Supported Python versions<\/td>\n<td>Python 3.10 and later<\/td>\n<td>Python 3.11\u20133.14 (Python 3.10 support dropped)<\/td>\n<\/tr>\n<tr>\n<td>Default CORS settings<\/td>\n<td>Allow all origins (<code>[\"*\"]<\/code>)<\/td>\n<td>Denied by default (empty list, explicit <code>--allowed-origins<\/code> required)<\/td>\n<\/tr>\n<tr>\n<td>Minimum dependency versions<\/td>\n<td><code>transformers&gt;=5.10.4<\/code><\/td>\n<td><code>transformers&gt;=5.16.1<\/code>, <code>tokenizers&gt;=0.23.2<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>max_tokens<\/code> memory reservation behavior<\/td>\n<td>Pre-reserved slots based on context length<\/td>\n<td>Treated as an upper bound, pre-reservation removed<\/td>\n<\/tr>\n<tr>\n<td>Removed features and paths<\/td>\n<td><code>mamba_cache_mode=\"all\"<\/code>, cascade attention, torch.compile SP\/AsyncTP paths<\/td>\n<td>Completely removed<\/td>\n<\/tr>\n<tr>\n<td>Responses API stream events<\/td>\n<td>Previous reasoning event names<\/td>\n<td>Changed to new stream event names<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>These changes impact users running Python 3.10 environments or production setups where vLLM servers are directly accessed from browsers via web frontends. In particular, due to the CORS specification change, systems integrating with browsers must explicitly specify <code>--allowed-origins<\/code> upon startup.<\/p>\n<h2>Key Changes<\/h2>\n<h3>Critical Security Vulnerability Fixes and Additional Defenses<\/h3>\n<p>A DoS vulnerability (GHSA-4hhp-h66f-j5j7) causing CPU and memory exhaustion via chat template rendering has been fixed, introducing static AST validation and timeout configuration (<code>VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT<\/code>). Additionally, countermeasures have been added for a bug where off-by-one errors in the tokenizer&#8217;s <code>max_token_id<\/code> caused CUDA assert failures (GHSA-q97c-p368-75c4), and a crash vulnerability caused by P2P ZMQ socket exhaustion (GHSA-frvg-qw88-hww9), introducing connection limits (<code>VLLM_P2P_MAX_PEERS<\/code>, default 128). Robustness has also been enhanced in multiple areas, such as restricting untrusted image loading to raster formats via an allowlist, HMAC signature verification for shared memory cache handles, and rejecting nested structured JSON schemas exceeding 64 levels (returning HTTP 400).<\/p>\n<h3>Expanded Support for Low-Precision KV Caches<\/h3>\n<p>The usage scope for low-precision KV caches, which improve memory efficiency, has been expanded. NVFP4 KV cache now supports NVIDIA SM8x, SM90, and SM12x, and is available for XQA decoding on SM12x and MiniMax-M3 sparse attention on SM100. Additionally, FP8 KV cache becomes the default for GLM-5.x in SM100 environments, improving throughput by 2.3\u20135.5%, while a 4-bit format <code>ultraquant_4bit<\/code> KV cache\u2014offering roughly double the density of FP8\u2014has been introduced for ROCm environments.<\/p>\n<h3>Model Runner V2 (MRV2) Enhancements and Default Adoption on ROCm<\/h3>\n<p>Model Runner V2, the next-generation runner, adds support for variable-length drafts in GPU n-gram speculative decoding, Elastic EP, combined pipeline parallelism and context parallelism for MLA models, and standard <code>torch.compile<\/code> mode. Furthermore, MRV2 is now set as the default runner for all models in ROCm environments.<\/p>\n<h3>Faster Startup and Improved Serving Performance<\/h3>\n<p>The new <code>vllm download-kernels<\/code> command allows users to install pre-compiled FlashInfer kernels, reducing cold-start times. Other additions include <code>--preload-autotune<\/code> to retain autotune tables across restarts, tensor arenas (<code>--enable-shm-tensor-arena<\/code>, enabled by default) to transfer multimodal tensors to workers via shared memory in a single copy, and loading tokenizers from local snapshots. Furthermore, in environments where <code>--api-server-count<\/code> is set to 2 or more, connections are now routed to the API worker with the lowest load, increasing requests per second by 43\u201385% for short requests.<\/p>\n<h2>Supported Models and Hardware<\/h2>\n<h3>Newly Added Models and Expanded Architectures<\/h3>\n<p>This version adds support for several new model architectures.<br \/>\n&#8211; <strong>EmbeddingGemma 2 (<code>google\/embeddinggemma-2<\/code>)<\/strong>: Supports omni-modal embedding models for text, image, audio, and video. By specifying <code>--limit-mm-per-prompt<\/code>, encoders for modalities not used within the prompt can be skipped for efficient execution.<br \/>\n&#8211; <strong>MiniCPM-V 4.7<\/strong>: Newly supported for serving.<br \/>\n&#8211; <strong>Nemotron 3.5 ASR<\/strong>: Supported as a streaming speech-to-text model.<br \/>\n&#8211; <strong>Structured Decision-Making Models<\/strong>: Supports execution of <code>Cloudflare\/clef-omni<\/code>, <code>LiquidAI\/d1-3B<\/code>, and DiffusionGemma via the newly established <code>\/v1\/systemone<\/code> endpoint.<br \/>\n&#8211; <strong>Others<\/strong>: Added support for PLaMo 3 inference and tool parsers, as well as the zerank-2 reranker score template.<\/p>\n<p>In addition, video input is now supported in the Transformers modeling backend, and models such as Glm, Arcee, CWM, Mellum, GPT-NeoX, Phi, Seed-OSS, and Jais2 can now be served via this backend.<\/p>\n<h3>Inference Optimizations and New Features for Existing Models<\/h3>\n<p>A number of architecture-specific optimizations have been introduced for major large-scale models.<br \/>\n&#8211; <strong>DeepSeek-V4 and V4.1<\/strong>: Features bounded replays for decoder-side SWA (Sliding Window Attention) and optimized replay layers within CUDA graphs. It introduces packed FP8 Engram exchanges across data parallelism (DP), faster Engram searches during CPU offloading, and drafting fusing DeepSeek-V4 with MTP (Multi-Token Prediction). Furthermore, encoder CUDA graph packing for V4.1 and Kimi-K3 reduces average TTFT (Time To First Token) by 11%.<br \/>\n&#8211; <strong>GLM-5.3 and GLM-5.3-Flash<\/strong>: Reusing sparse MLA (Multi-head Latent Attention) index transformations across layers sped up the corresponding kernel by 3.5\u20133.9x, and fused multi-step draft decoding was added. Support for <code>fp8_ds_mla<\/code> on SM90, a 4.4\u20137.7% TTFT improvement via skipped qlnorm for MHA, and indexer prefill distribution via tensor parallelism at 1-million context (reducing TTFT by up to 25%) are also included.<br \/>\n&#8211; <strong>Qwen Family<\/strong>: Qwen3.8-Flash-Next (Qwen4Exp) integrates QSA projections into a single GEMM, achieving a 1.45\u20131.68x speedup. Sequence parallelism support via <code>--enable-hc-sp<\/code> and low-latency decoding GEMM plans for SM100 (up to 2.7x faster than cuBLAS for small M sizes) are also provided. Additionally, M-RoPE cache sizes for Qwen3-VL, Qwen3.5, and Qwen3.8 have been reduced to a quarter.<br \/>\n&#8211; <strong>Kimi-K3 and MiniMax-M3<\/strong>: FlashInfer fused speculative KDA on SM100 and a fused vision processor improving preprocessing speed by 10\u201320x at 1080p\u20138K resolutions have been added for Kimi-K3. For MiniMax-M3, MSA prefill combining FP8 KV cache and E4M3 queries is 26\u201337% faster.<br \/>\n&#8211; <strong>General Multimodal<\/strong>: Device-side input normalization was introduced for Qwen3-VL, Qwen3.5, Qwen4Exp, GLM-V, GLM-OCR, Llama Nemotron VL, and Kimi K2.5\/K3, sorting encoder inputs by modality to boost request processing capacity by 6.5\u20138.2%.<\/p>\n<h3>Hardware Platform Enhancements<\/h3>\n<p>Significant performance enhancements have been made not only for NVIDIA GPUs, but also for AMD ROCm, Intel XPU, and CPU environments.<br \/>\n&#8211; <strong>NVIDIA<\/strong>: SM100 now supports native 64-head FlashMLA sparse prefill, improving TTFT by 1.16\u20131.18x. Split-K attention for SM120 and OAI Triton MXFP4 MoE kernels for SM12x have also been added.<br \/>\n&#8211; <strong>AMD ROCm<\/strong>: AITER MegaMoEV2 via <code>--moe-backend aiter_mega_moe<\/code> was introduced for DeepSeek-V4, with support for shared expert combination (3\u20138% improvement). Mono decode (<code>VLLM_ROCM_MONO_DECODE=1<\/code>) for DeepSeek-V4.1-Flash on MI355X, Gluon sparse MLA for GLM-5.2 (up to 16% improvement), <code>ROCM_SEGMENTED_ATTN<\/code> for RDNA, and W4A16 tuning on gfx11 (up to 23% TTFT improvement) are also available.<br \/>\n&#8211; <strong>Intel XPU<\/strong>: MLA support in <code>FLASH_ATTN_MLA<\/code>, FP8 GEMMs (2.0\u20132.73x faster) and fused MoE configurations for Intel Arc Pro B70, and fused SYCL LayerNorm have been introduced.<br \/>\n&#8211; <strong>CPU<\/strong>: Vectorized samplers for s390x environments (up to 58x), causal conv1d optimizations for aarch64 (approx. 9% improvement), and AWQ\/GPTQ support for POWER10 have been added.<\/p>\n<h3>Quantization Feature Expansion<\/h3>\n<p>On the quantization front, online requantization of linear layers (<code>--quantization-config.linear fp8_per_channel<\/code>) is now possible, alongside token-wise NVFP4 CuTe-DSL MoE and MXFP4 MoE oracles equipped with CUTLASS W4A4 fallbacks. ModelOpt&#8217;s IQ2_XS, IQ2_XXS, and Q8_0 formats are also supported for SM120\/SM121.<\/p>\n<h2>How to Get It<\/h2>\n<p>When updating to vLLM v0.32.0, ensure that the Python version in your execution environment is 3.11\u20133.14 beforehand.<\/p>\n<p>Update packages using the recommended <code>uv<\/code> or standard <code>pip<\/code> commands. Running <code>vllm download-kernels<\/code> after installation deploys pre-compiled kernels and reduces startup time.<\/p>\n<pre><code class=\"language-bash\">uv pip install vllm\nvllm download-kernels\n<\/code><\/pre>\n<p>If using standard <code>pip<\/code>, execute the following:<\/p>\n<pre><code class=\"language-bash\">pip install vllm\nvllm download-kernels\n<\/code><\/pre>\n<p>For ROCm and Intel XPU environments, install by specifying their respective dedicated extra index URLs.<\/p>\n<pre><code class=\"language-bash\"># For AMD ROCm environments\npip install vllm --extra-index-url https:\/\/wheels.vllm.ai\/rocm\/0.32.0\/rocm723\n\n# For Intel XPU environments\nuv pip install vllm --extra-index-url https:\/\/wheels.vllm.ai\/0.32.0\/xpu --extra-index-url https:\/\/download.pytorch.org\/whl\/xpu --index-strategy unsafe-best-match\n<\/code><\/pre>\n<p>When using Docker containers, pull and update to the official Docker images.<\/p>\n<pre><code class=\"language-bash\"># CUDA 13.0 environment (default)\ndocker pull vllm\/vllm-openai:v0.32.0\n\n# CUDA 12.9 environment\ndocker pull vllm\/vllm-openai:v0.32.0-cu129\n\n# AMD ROCm environment\ndocker pull vllm\/vllm-openai-rocm:v0.32.0\n\n# CPU environment\ndocker pull vllm\/vllm-openai-cpu:v0.32.0\n\n# Intel XPU environment\ndocker pull vllm\/vllm-openai-xpu:v0.32.0\n<\/code><\/pre>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/vllm-v0-34-0-released\/\">vLLM v0.34.0 Released with Extensible KV Cache and New Models<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/vllm-v033-0-released\/\">vLLM v0.33.0 Released: CUDA 13.0, Speculative Decoding Upgrades<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/05\/vllm-v0310-released\/\">vLLM v0.31.0 Released: Fast Restart and Hardware Optimization<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/22\/vllm-v0-30-0-released\/\">vLLM v0.30.0 Released: Fast Start Weight Caching and New Models<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Follow this tool<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-vllm-en\/\">vLLM overview and release history (101 releases tracked)<\/a><\/li>\n<li><strong>Other inference engines and runtimes<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-ollama-en\/\">Ollama<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-sglang-en\/\">SGLang<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/releases\/tag\/v0.32.0\">https:\/\/github.com\/vllm-project\/vllm\/releases\/tag\/v0.32.0<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/50300\">https:\/\/github.com\/vllm-project\/vllm\/pull\/50300<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/59491\">https:\/\/github.com\/vllm-project\/vllm\/pull\/59491<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/51504\">https:\/\/github.com\/vllm-project\/vllm\/pull\/51504<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/61009\">https:\/\/github.com\/vllm-project\/vllm\/pull\/61009<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/60022\">https:\/\/github.com\/vllm-project\/vllm\/pull\/60022<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/59357\">https:\/\/github.com\/vllm-project\/vllm\/pull\/59357<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/60036\">https:\/\/github.com\/vllm-project\/vllm\/pull\/60036<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/vllm-project\/vllm\/pull\/60402\">https:\/\/github.com\/vllm-project\/vllm\/pull\/60402<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>vLLM v0.32.0 is now available, featuring Python 3.11+ requirement, low-precision KV cache support, security fixes, and hardware acceleration.<\/p>\n","protected":false},"author":1,"featured_media":12415,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[1133,163,165,1547,592],"class_list":["post-12416","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-fp8-en","tag-gguf-en","tag-moe-en","tag-verified","tag-vllm-en"],"lang":"en","translations":{"en":12416,"ja":12414},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/12416","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=12416"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/12416\/revisions"}],"predecessor-version":[{"id":17853,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/12416\/revisions\/17853"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/12415"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=12416"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=12416"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=12416"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}