vLLM v0.30.0 Released: Fast Start Weight Caching and New Models

vLLM v0.30.0 Released: Fast Start Weight Caching and New Models

At a Glance

Item Value
Repository vllm-project/vllm
Version v0.30.0
Published 2026-09-22
License Apache-2.0
Source type Primary source (the publisher itself)

Values determined by this site’s code at collection time. Dates are JST.

Overview

vLLM v0.30.0 has been released. vLLM is an open-source engine designed for fast and memory-efficient LLM inference and serving, developed under the Apache-2.0 license. With over 90,000 stars on GitHub, it has become a standard choice for running LLMs in local environments and cloud servers.

The most significant impact for users in this version is the introduction of “Fast Start (weight caching)", which dramatically reduces engine restart times. This makes it possible to keep model weights resident in GPU memory, significantly improving the efficiency of trial and error in development and operations.

Breaking Changes and Deprecations

This version introduces migrations to command-line arguments and removes legacy environment variables. Users of scale-out features or gRPC servers will need to update their startup commands.

Old Setting / Method New Setting / Method
VLLM_ENABLE_SCALE_OUT_ENDPOINTS (environment variable) --enable-scale-out (argument)
python -m vllm.entrypoints.grpc_server vllm serve --grpc
VLLM_PREFIX_CACHE_RETENTION_INTERVAL (environment variable) (Removed)
VLLM_MM_HASHER_ALGORITHM (environment variable) (Removed)
GPTQ g_idx (activation ordering) (Removed)
Mamba all cache mode (Deprecated)

These changes affect users launching API servers with vllm serve, those utilizing specific quantization models, and those using the Mamba architecture. Additionally, because YaRN (Yet another RoPE extensioN) behavior has been unified with Transformers, max_model_len is no longer automatically rescaled when using vendor-specific aliases.

Key Changes

Fast Start: GPU Memory Weight Caching

The new “Fast Start" feature speeds up engine restarts. It works by having a persistent per-GPU weight cache daemon keep quantized and tensor-parallel (TP) split weights in GPU memory. Rather than reloading from disk upon engine restart, specifying --load-format ipc_cache allows mapping weights directly from memory via CUDA IPC. This feature also supports FP4 checkpoints and multi-node TP, proving powerful in development scenarios that require frequent restarts.

Engine Initialization and Graph Capture Speedups

Improvements to Model Runner V2 have significantly reduced engine initialization times. Notably, optimizations that freeze garbage collection (GC) during CUDA graph capture have accelerated capture times from 12 seconds to 2 seconds on H200 environments, and total engine initialization from 28.9 seconds to 8.2 seconds. Furthermore, numerous changes enhancing inference efficiency are included, such as dual-batch overlap in eager mode and full CUDA graph support in micro-batch steps.

Qwen3.8-Flash-Next Optimizations and Memory Savings

Dedicated Triton kernels have been introduced for Qwen3.8-Flash-Next (Qwen4Exp). Performance has been improved through separate prefill and decode paths for the QSA indexer, fused PLE kernels, and an FP8 indexer cache. Additionally, removing torch.compile in the NVIDIA implementation enables loading FP8 checkpoints on a single GB300. Previously, torch.compile auto-tuning consumed about 50GB of extra memory, causing OOM (Out of Memory) errors, which is now resolved. It is also now possible to offload PLE weights to the CPU via --engram-config and read them using CUDA UVA (Unified Virtual Addressing).

DeepSeek-V4.1-Flash and MLA Enhancements

Support for the latest DeepSeek-V4.1-Flash has been added. On SM100 environments, FlashMLA V4.1 can be utilized to keep the entire KV cache in MXFP8 format. Additionally, a “HiSparse" feature has been introduced to evict KV pages to host memory (RAM) when GPU memory is under pressure. This can be enabled via HiSparseConnector, improving flexibility when handling massive contexts by combining hot buffers on the GPU with host caches.

Quantization and Watermarking

Targeted online quantization of specific layers is now possible via the quantization configuration quantization_config.targets. CUDA support for 2/3/5/6/7-bit quantization via AutoRound and improvements to the NVFP4 format have also been implemented. On the generation side, the engine now supports the generation and detection of watermarks using Gumbel-max. This can be opted out per inference request and is compatible with speculative decoding.

Supported Models and Hardware

The new v0.30.0 version adds support for many latest model architectures and hardware-specific features. Newly supported models include DeepSeek-V4.1-Flash (with support for FlashMLA V4.1 MXFP8 KV cache storage and DeepGEMM Mega-mHC, etc.), DeepSeek-V4-Flash-Vision-Exp (including ROCm support and LoRA), GLM-5.3-Flash (with EPLB support), K2-Horizon with reasoning and tool parsers, Cohere Compass, Bailing V3 VL with MTP support, Nanbeige4.2 via the Transformers backend, and the DeepSeek-V4 CPU backend with AVX512/AMX support.

In terms of hardware optimization, a public CUDA 13.4 Rubin build path has been prepared for NVIDIA environments, alongside block-level FP8 CTA raster swizzling for SM12x (for GB10/DGX Spark), prioritization of W4A4 NVFP4 on SM120/121, and faster graph capture on H200. For AMD ROCm environments, the TheRock base has been updated to ROCm 10.0, including the adoption of AITER 0.1.21.post2, DeepSeek V4 shared expert fusion, MXFP8 decant optimizations, and several other performance improvements.

How to Get It

Use uv (recommended) or pip to install and update vLLM. Run the appropriate command below according to your GPU environment and purpose:

uv pip install vllm

Refer to the official documentation if you are building from source for development. For environments other than CUDA 13.0, or when using ROCm and XPU, please use platform-specific additional index URLs or Docker images (such as vllm/vllm-openai:v0.30.0).

Related Articles

Sources

Engines and ToolsPython,Verified,vLLM,推論エンジン