SGLang v0.5.23 Released: Major Performance and Feature Updates

At a Glance
| Item | Value |
|---|---|
| Repository | sgl-project/sglang |
| Version | v0.5.23 |
| Published | 2026-11-03 |
| License | Apache-2.0 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
SGLang is a high-performance serving framework for large language models (LLMs) and multimodal models. Its latest version, v0.5.23, has been released.
The main highlight of this update is the dramatic improvement in inference performance. In particular, it is reported that when running DeepSeek-V4.1 on GB300, the prefill throughput under load is improved by up to 1.5x, and the time-to-first-token latency for 32K to 64K prompts is reduced by approximately 28%.
Breaking Changes and Deprecations
Several changes have been made that affect existing configurations and commands.
| Item | Old | New |
|---|---|---|
| Inference option | --speculative-skip-dp-mlp-sync |
Removed (will cause an error if used) |
| Attention backend | tokenspeed_mla |
cutedsl_mla (tokenspeed_mla remains as a deprecated alias) |
The --speculative-skip-dp-mlp-sync option has been removed because this flag caused servers to crash under specific configurations when Attention DP was in use. Environments currently using DP attention should run without specifying this flag.
Additionally, the tokenspeed_mla backend has been removed, and cutedsl_mla now takes its place. If existing code specifies tokenspeed_mla, a warning will be displayed and it will be mapped to cutedsl_mla, but users are encouraged to explicitly migrate to the new backend name.
Key Changes
Substantial Inference Performance Improvements
For MoE (Mixture-of-Experts) models such as DeepSeek-V4.1 using H200, the first token generation is 30% to 58% faster for 8K prompts. Furthermore, the “low-latency mode," which optionally utilizes NVIDIA’s megakernel and AMD’s monokernel plugins, accelerates the decode speed of DeepSeek-V4.1-Flash by up to 44% on GB300 and up to 75% on MI350X. These improvements provide massive benefits to users looking to leverage the latest GPU hardware for low-latency inference.
Memory Savings and Speedups via Paged Experts
The optional “Paged Experts" feature now allows MoE models that exceed GPU memory capacity to run on a single consumer-grade GPU. This feature operates up to 30x faster compared to CPU offloading, serving as an extremely effective option for users who want to run massive models with limited VRAM resources.
Multimodal and Diffusion Model Optimizations
Image and video generation models such as Cosmos3, Qwen-Image, MOVA, and HunyuanVideo have received optimizations that reduce host syncs within the denoising loop and per-step data transfers. This reduces the time the host spends waiting while the GPU performs computations, improving the efficiency of the overall generation process. Users running image and video generation models on SGLang can benefit from these model-specific optimizations.
Support for New Hardware Platforms
Support for Google TPU7x has been introduced, realized through the PyTorch-native plugin “SGLang-TorchTPU." A new optional LoRA engine has also been introduced, enabling the serving of LoRA adapters for quantized MoE models (FP8 and NVFP4). This is an important update for engineers looking to deploy new hardware or apply LoRA to quantized models.
Supported Models and Hardware
This version adds support for several new LLM, multimodal model, and diffusion model architectures and quantization methods.
- LLM / VLM: Muse Spark Open, Beam-1.0, MiniMax-M3.1-Flash, MiniCPM-V 4.7, Kev
- Diffusion: Cosmos3-Nano-Sim (Bimanual, Transfer, Auto), FastH3 Omni, FLUX.3 Image Klein, HY-Nomas, MOSS-Image
- Quantized Models: Added the MiniMax-M3.1 NVFP4 MoE method, and enabled serving LoRA adapters for quantized MoE models (FP8 and NVFP4).
- Hardware: Added support for Google TPU7x via “SGLang-TorchTPU," supporting execution in Google TPU environments in addition to NVIDIA and AMD.
These models can now undergo GGUF conversion, inference, and cookbook usage, expanding the choices of supported hardware.
How to Get It
For installation in a development environment, use the following command:
uv pip install --prerelease=allow -e "python"
When setting up an environment using Docker, the following images prepared for each platform can be utilized:
- NVIDIA (CUDA 13):
docker pull lmsysorg/sglang:v0.5.23 - AMD MI35x:
docker pull lmsysorg/sglang:v0.5.23-rocm10-mi35x - AMD MI30x:
docker pull lmsysorg/sglang:v0.5.23-rocm10-mi30x - Intel GPU:
docker pull lmsysorg/sglang:v0.5.23-xpu - Intel CPU:
docker pull lmsysorg/sglang:v0.5.23-xeon - Google TPU7x (TorchTPU):
docker pull lmsysorg/sglang:v0.1.0-tpu
Related Articles
- SGLang v0.5.20 Released: New Models and Optimizations
- SGLang v0.5.22 Released: 10x Faster Loading and DeepSeek-V4.1 Boost
- SGLang v0.5.21 Released with Dynamic PD Role Switching
- DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds
What to Read Next
- Follow this tool → SGLang overview and release history (63 releases tracked)
- Other inference engines and runtimes → llama.cpp / Ollama / vLLM
Sources
- sgl-project/sglang v0.5.23 Release Notes
- Pull Request #45043: Remove –speculative-skip-dp-mlp-sync
- Pull Request #44803: Remove the tokenspeed_mla backend and dependency
- Pull Request #45432: Drop the fixed 64-item multimodal cap in favor of –limit-mm-data-per-request
- Pull Request #45967: Remove per-step and per-request host syncs in Cosmos3 and Qwen-Image-2.1
- Pull Request #46046: Remove per-step host syncs and uploads in image models
- Pull Request #46047: Remove per-step host syncs and uploads in video models
- Pull Request #45170: Remove blocking seq_lens D2H from graph replay
- Pull Request #44393: Move the remaining CUDA build paths off Python 3.10

