{"id":1848,"date":"2026-09-19T08:09:11","date_gmt":"2026-09-18T23:09:11","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/19\/sglang-v0-5-20-released\/"},"modified":"2026-09-19T08:09:11","modified_gmt":"2026-09-18T23:09:11","slug":"sglang-v0-5-20-released","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/19\/sglang-v0-5-20-released\/","title":{"rendered":"SGLang v0.5.20 Released: New Models and Optimizations"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/github.com\/sgl-project\/sglang\">sgl-project\/sglang<\/a><\/td>\n<\/tr>\n<tr>\n<td>Version<\/td>\n<td><a href=\"https:\/\/github.com\/sgl-project\/sglang\/releases\/tag\/v0.5.20\">v0.5.20<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-19<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>Apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code at collection time. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>SGLang v0.5.20, a high-performance serving framework for large language models and multimodal models, has been released. This version includes numerous changes from 713 PRs and 237 contributors, bringing support for new models and various feature enhancements.<\/p>\n<p>The most impactful change for users is that the persistence behavior of the <code>\/v1\/responses<\/code> API has been disabled by default and changed to an opt-in basis. Systems using this feature will require explicit configuration when starting the server.<\/p>\n<h2>Breaking Changes and Deprecations<\/h2>\n<ul>\n<li>Changes to Responses API persistence: The default value has been changed from enabled to disabled.<\/li>\n<\/ul>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Old Behavior<\/th>\n<th>New Behavior<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Unconditionally holds <code>\/v1\/responses<\/code> results in memory<\/td>\n<td>Holds results only when <code>--enable-response-store<\/code> is specified at server startup<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<ul>\n<li>\n<p>Who is affected: Users utilizing the <code>\/v1\/responses<\/code> endpoint for retrieval, chaining via <code>previous_response_id<\/code>, or background requests. Running without the flag will return a 400 error (cannot be enabled in PD deployments).<\/p>\n<\/li>\n<li>\n<p>Removal of Prefill Context Parallelism (CP) v1: The legacy implementation and related CLI options have been removed and consolidated into the strategy-based implementation. Prefill CP on HIP, NPU, and MUSA platforms is rejected until porting to each platform is complete.<\/p>\n<\/li>\n<\/ul>\n<h2>Key Changes<\/h2>\n<p><strong>RL Rollout Support via Sampling Mask<\/strong> Using <code>return_sampling_mask<\/code> now returns the exact token support referenced by the sampler at each step of decoding, along with the log probabilities of the sampled tokens under it. This allows trainers to reproduce rollouts without reconstructing top-k or top-p. Operating under overlap scheduling, Qwen3-8B shows a 17% decode throughput improvement at batch 1 and 52% at batch 64 compared to the previous version. Capacity is configured with <code>--sampling-mask-max-tokens<\/code> (default 4096).<\/p>\n<p><strong>Efficient Caching via Unified Radix Tree<\/strong> Branch point caching in SWA components now maintains sliding window state at points where requests diverge from a shared prefix. This allows the branching side to reuse the cache without recalculation. In the DeepSeek-V4-Flash shared prompt environment, the token hit rate increased from 43.8% to 60.8%, and the average TTFT decreased from 1.57 seconds to 1.07 seconds.<\/p>\n<p><strong>Faster Model Loading in ROCm Environments<\/strong> Large pageable host-device copies were changed from on-the-fly pinning to a staging approach. This resolves the issue where the driver suspends the GPU queue on every eviction, reducing the load time for GLM-5.2 with TP4 on 4x MI355X from 505.7 seconds to 40.4 seconds.<\/p>\n<p><strong>Provision of Intel XPU Official Release Images<\/strong> For all tagged releases, <code>lmsysorg\/sglang:vX.Y.Z-xpu<\/code> built from the Dockerfile for XPU is now published. This allows Intel GPU users to use version-pinned images without relying on nightly builds.<\/p>\n<h2>Supported Models and Hardware<\/h2>\n<p>GGUF conversion and inference are now possible for new autoregressive models GLM-5.3-Flash, Hy4-Preview, Qwen3.8-Flash-Next, K2 Horizon, and Nanbeige4.2, as well as diffusion models SenseNova-U1.5-8B-MoT, FastH3 (4-step MiniMax-H3 distillation), and VDN-H3 (hybrid attention MiniMax-H3 distillation). It also includes DeepSeek-V4 optimizations for the Blackwell architecture and improved support for RTX PRO 6000 environments.<\/p>\n<h2>How to Get It<\/h2>\n<p>Please refer to the official installation instructions and documentation to perform the update.<\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/sgl-project\/sglang\/releases\/tag\/v0.5.20\">sgl-project\/sglang v0.5.20<\/a><\/li>\n<li><a href=\"https:\/\/docs.sglang.io\/get_started\/install.html\">SGLang Getting Started<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>SGLang v0.5.20 is released with 713 PRs, adding support for new models, ROCm load time improvements, Intel XPU official images, and breaking changes.<\/p>\n","protected":false},"author":1,"featured_media":1847,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[604,163,520,169,1547],"class_list":["post-1848","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-deepseek-en","tag-gguf-en","tag-qwen-en","tag-sglang-en","tag-verified"],"lang":"en","translations":{"en":1848,"ja":1846},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1848","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=1848"}],"version-history":[{"count":0,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1848\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/1847"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=1848"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=1848"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=1848"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}