{"id":9028,"date":"2026-10-02T11:11:13","date_gmt":"2026-10-02T02:11:13","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/10\/02\/sglang-v0-5-21-released-2\/"},"modified":"2026-10-02T11:11:13","modified_gmt":"2026-10-02T02:11:13","slug":"sglang-v0-5-21-released-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/02\/sglang-v0-5-21-released-2\/","title":{"rendered":"SGLang v0.5.21 Released with Dynamic PD Role Switching"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nsgl-project\/sglang<\/a><\/td>\n<\/tr>\n
Version<\/td>\nv0.5.21<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-10-02<\/td>\n<\/tr>\n
License<\/td>\nApache-2.0<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code when the information was collected. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

SGLang is a high-performance serving framework for large language models (LLMs) and multimodal models. Version 0.5.21 has been released in this update.<\/p>\n

The most significant change in this release is that PD (Prefill\/Decode) instances can now dynamically switch between Prefill and Decode states during execution without restarting. This is expected to significantly improve the utilization efficiency of inference resources.<\/p>\n

Breaking Changes & Deprecations<\/h2>\n

This update removes features that were deprecated in past releases as well as experimental features. Caution is required in environments using these.<\/p>\n

\n\n\n\n\n\n\n\n\n
Change<\/th>\nOld State \/ Setting<\/th>\nNew State<\/th>\n<\/tr>\n<\/thead>\n
Removal of experimental C++ radix tree<\/td>\nSGLANG_EXPERIMENTAL_CPP_RADIX_TREE<\/code> environment variable<\/td>\nUnavailable<\/td>\n<\/tr>\n
Removal of specific Radix Cache implementations<\/td>\nSWARadixCache<\/code> \/ MambaRadixCache<\/code><\/td>\nIntegrated into UnifiedRadixCache<\/code><\/td>\n<\/tr>\n
Removal of unused HiRadixCache<\/td>\nHiRadixCache<\/code><\/td>\nUnavailable<\/td>\n<\/tr>\n
Removal of past deprecated endpoints<\/td>\nAPIs\/environment variables\/aliases deprecated for 2+ releases<\/td>\nUnavailable<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

These primarily affect developers and users operating with specific experimental cache settings explicitly specified.<\/p>\n

Key Changes<\/h2>\n

Dynamic Role Switching for PD Instances<\/h3>\n

In PD (Prefill\/Decode) instances, Prefill and Decode roles can now be switched on-the-fly without restarting the instance. This enables flexible adaptation to changes in inference workloads.<\/p>\n

Rust-Based Core for Prefix Cache<\/h3>\n

Prefix Cache now operates on a Rust-based core by default. This aims to improve cache management efficiency and performance.<\/p>\n

Introduction of New APIs (Decisions API \/ Score API)<\/h3>\n

New APIs have been added to utilize LLMs and VLMs as low-latency classifiers or scorers.
\n– \/v1\/decisions<\/code> (Decisions API): Functions the model as a low-latency classifier or scorer.
\n– \/v1\/score<\/code> (Score API): Calculates scores for all candidates in a single request.<\/p>\n

Performance Improvements for Specific Models<\/h3>\n