SGLang v0.5.22 Released: 10x Faster Loading and DeepSeek-V4.1 Boost

- 1. At a Glance
- 2. Overview
- 3. Breaking Changes and Deprecations
- 4. Key Changes
- 4.1. Faster Model Loading via InstantTensor Loader
- 4.2. Enhanced Prefill/Decode Disaggregation (PD Disaggregation)
- 4.3. Standard PyTorch Backend Support on Apple Silicon Macs
- 4.4. Significant Inference Throughput Improvements for Latest Model Groups
- 4.5. Speculative Decoding Support with HiSparse for Long Contexts
- 4.6. Critical Bug Fix Regarding Speculative Decoding and Structured Output Combination
- 5. Supported Models and Hardware
- 6. How to Get It
- 7. Related Articles
- 8. What to Read Next
- 9. Sources
At a Glance
| Item | Value |
|---|---|
| Repository | sgl-project/sglang |
| Version | v0.5.22 |
| Published | 2026-10-19 |
| License | Apache-2.0 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
SGLang version v0.5.22, a high-performance inference serving framework for large language models and multimodal models, has been released. SGLang is an Apache-2.0 licensed open-source project developed primarily in Python, known for its fast KV cache management and diverse parallel processing backends.
The most significant impacts for users in this version are a substantial increase in inference throughput for latest models such as DeepSeek-V4.1, and the introduction of the new “InstantTensor" loader, which reduces model loading times by up to 10x. Additionally, the required Python version has been raised to 3.11 or later, meaning a pre-update runtime upgrade may be necessary depending on the environment.
Breaking Changes and Deprecations
In this release, deprecations and renames have been made regarding parallelism-related startup options and environment variable names. Checking configuration values prior to updating is required.
| Item | Old Setting | New Setting | Migration Details & Default |
|---|---|---|---|
| Attention DP Specification | --enable-dp-attention |
--attn-dp-size <N> (alias --attention-data-parallel-size) |
Previously used in combination with --dp-size to specify the number of attention data parallel groups, but now separated into a dedicated option. The default is 1. |
| Model Replica Specification | --dp-size |
--dp-size (standalone use) |
Streamlined to specify only the model replica count. Using --dp-size > 1 and --attn-dp-size > 1 simultaneously will result in an error. |
| SWA Prefill Unlock | SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW |
SGLANG_OPT_RELEASE_PREFILL_SWA |
The environment variable name has been changed to match actual behavior. The old name will also work as a deprecated alias for the time being. |
| Required Python Version | Python 3.10 or earlier | Python 3.11 or later | The execution requirement for SGLang has been raised to Python 3.11 or later. |
This change affects environments utilizing attention data parallelism within Tensor Parallelism (TP), or users who embed conventional environment variables into startup scripts. Furthermore, if SGLang is running in a Python 3.10 environment, the execution environment’s Python version must be updated before upgrading.
Key Changes
Faster Model Loading via InstantTensor Loader
The InstantTensor loader has been introduced as a new feature that significantly revamps weight loading processing, speeding up model load times by up to 10x. This dramatically reduces startup wait times for developers who frequently restart and test LLMs with large weight files, or in production environments spinning up instances on-demand. No re-conversion or re-downloading of model files is required to use it.
Enhanced Prefill/Decode Disaggregation (PD Disaggregation)
In the PD disaggregation architecture, which operates by separating prefill and decode instances, workloads handling long prompts achieved up to a 27% throughput increase and halved the p95 TTFT (Time To First Token). This directly benefits infrastructure operators hosting long-context processing at scale. Furthermore, efficiency in distributed transfers has advanced, such as NIXL being accepted as a transfer backend for DSpark PD in DeepSeek-V4.1.
Standard PyTorch Backend Support on Apple Silicon Macs
In Mac environments powered by Apple Silicon, execution is now possible using the standard PyTorch backend in addition to the conventional MLX backend. This broadens the available backend choices for engineers looking to directly test and develop SGLang’s inference pipeline and features locally on a Mac.
Significant Inference Throughput Improvements for Latest Model Groups
Kernel optimizations and graph fusions were implemented for numerous models across various NVIDIA and AMD GPU environments. Per-GPU throughput for DeepSeek-V4.1 on NVIDIA H200 improved by up to 2.3x, and throughput up to 1.9x was achieved using DSpark in AMD MI350X and MI355X environments. Additionally, decode speed for Qwen3.8-Flash-Next in NVIDIA B200 environments became 12% faster, and MiMo-V2.6-Pro recorded 1.85x decode throughput via DFlash in B300 environments.
Speculative Decoding Support with HiSparse for Long Contexts
In HiSparse, which offloads the KV cache to CPU memory to process long contexts, speculative decoding via EAGLE and MTP is now supported. Engineers performing long-context serving in limited GPU memory environments can improve decode speed while suppressing memory consumption.
Critical Bug Fix Regarding Speculative Decoding and Structured Output Combination
An issue where executing a hybrid linear model with both speculative decoding and structured outputs simultaneously caused it to resume from incorrect Mamba states has been fixed. Reported by Mr. Wirick and Baseten, this prevents output corruption when performing context generation or constrained generation such as JSON on models adopting Mamba or hybrid structures. Upgrading early is recommended if using related models.
Supported Models and Hardware
In this version, supported models have been expanded across a wide range, not only for text and multimodal models, but also diffusion models responsible for image and video generation. Optimizations for the latest GPU architectures and platform extensions have also progressed.
Newly Supported Models (LLM / VLM)
Inference serving for the following models has been newly supported as language models and vision-language models (VLMs):
- Clef / Clef Flash: Models by Cloudflare, capable of native inference on SGLang.
- PPLX-Decider v1 / v1.1: Added support for model groups published by Perplexity.
- EmbeddingGemma 2: Supported Google’s text embedding model, enabling fast embedding vector extraction.
- LLaDA2.2: Includes support for the Block Routing MoE architecture.
Newly Supported Models (Diffusion) and ComfyUI Integration
Various generative AI image and video models are now available in the SGLang Diffusion engine:
- Image Generation Models: HunyuanImage-3.0-Instruct, Kandinsky 6 / Kandinsky 6 SR, LLaDA-Image / LLaDA-Image-Turbo
- Video Generation Models: Wan-Animate-2, SANA-Video 2.0
Furthermore, integration between the image generation tool ComfyUI and SGLang Diffusion has progressed, making MiniMax-H3 and Qwen-Image 2.1 runnable on ComfyUI. Capabilities to directly load and execute community-made GGUF checkpoints or LoRAs via SGLang Diffusion are also included.
Supported Hardware and Accelerators
On the hardware front, support for NVIDIA’s next-generation architecture, AMD accelerators, and Apple Silicon has been significantly enhanced.
- NVIDIA Blackwell (B200 / B300) and Hopper (H200): Various kernels for the SM100 generation (FA4, ptx_kda, etc.) are optimized, such as a 12% speedup for Qwen3.8-Flash-Next in B200 environments, a 10% speedup for GLM-5.3-Flash (when using DFlash) in B300 environments, and 1.85x throughput for MiMo-V2.6-Pro (when using DFlash). CUDA 13 environments are provided in the official Docker images.
- AMD Instinct MI350X / MI355X / MI300 Series: Containers targeting ROCm 10 environments are provided, allowing DeepSeek-V4.1 to run with up to 1.9x throughput utilizing DSpark.
- Apple Silicon Macs: Inference execution using the standard PyTorch backend is officially supported in addition to the conventional MLX backend.
- Intel GPUs / CPUs: XPU images for Intel GPUs and Docker containers optimized for Intel Xeon CPUs are provided respectively.
How to Get It
SGLang v0.5.22 requires Python 3.11 or higher. Please verify your running environment’s Python version before performing the update.
To install or update packages using uv, run the following command:
uv pip install --prerelease=allow sglang==0.5.22
When using Docker, please pull the appropriate official image based on your hardware environment:
- NVIDIA GPU (CUDA 13 Environment):
docker pull lmsysorg/sglang:v0.5.22
- AMD Instinct MI35x Series (ROCm 10 Environment):
docker pull lmsysorg/sglang:v0.5.22-rocm10-mi35x
- AMD Instinct MI30x Series (ROCm 10 Environment):
docker pull lmsysorg/sglang:v0.5.22-rocm10-mi30x
- Intel GPU (XPU):
docker pull lmsysorg/sglang:v0.5.22-xpu
- Intel Xeon CPU:
docker pull lmsysorg/sglang:v0.5.22-xeon
To set up the repository in editable mode as a development environment, start the development container lmsysorg/sglang:dev and run one of the following commands in the root directory of the repository:
pip install -e "python"
If a virtual environment is activated, the following command can also be used:
uv pip install --prerelease=allow -e "python"
Related Articles
- SGLang v0.5.23 Released: Major Performance and Feature Updates
- SGLang v0.5.21 Released with Dynamic PD Role Switching
- SGLang v0.5.20 Released: New Models and Optimizations
What to Read Next
- Follow this tool → SGLang overview and release history (63 releases tracked)
- Other inference engines and runtimes → llama.cpp / Ollama / vLLM
- How to read Perplexity → Benchmark glossary
Sources
- sgl-project/sglang v0.5.22 Release Notes
- PR #41818: [Feature] Add –attn-dp-size and deprecate –enable-dp-attention
- PR #40453: Models load 10x faster with the new InstantTensor loader
- PR #40703: PD disaggregation throughput and TTFT improvements
- PR #36780: SGLang on Apple Silicon Macs with standard PyTorch backend
- PR #41251: DeepSeek-V4.1 H200 throughput improvements
- PR #41308: DeepSeek-V4.1 AMD MI350X / MI355X DSpark support
- PR #37771: HiSparse speculative decoding support with EAGLE and MTP
- PR #43029: Fix mamba states resumption with speculative decoding and structured output
- PR #42657: Rename SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW
- PR #31768: Add LLaDA2.2 Block Routing MoE support
- PR #43504: MiniMax-H3 support in ComfyUI with SGLang Diffusion
- PR #43581: Qwen-Image 2.1 support in ComfyUI with SGLang Diffusion
- PR #42556: SGLang Diffusion GGUF checkpoint loader

