{"id":2980,"date":"2026-09-23T09:21:06","date_gmt":"2026-09-23T00:21:06","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/23\/together-ai-canary-rollouts-2\/"},"modified":"2026-09-23T09:21:06","modified_gmt":"2026-09-23T00:21:06","slug":"together-ai-canary-rollouts-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/23\/together-ai-canary-rollouts-2\/","title":{"rendered":"Together AI Announces Canary Rollouts for Zero-Downtime Updates"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Publisher<\/td>\nTogether AI<\/td>\n<\/tr>\n
Published<\/td>\n2026-09-22<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code at collection time. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

Together AI has released “Canary Rollouts,” a phased update methodology for zero-downtime model updates in production. This method gradually migrates traffic from a current model (Source) to a new checkpoint or a new model family (Target) according to defined steps.<\/p>\n

Claims and Evidence<\/h2>\n

Together AI claims that by using Canary Rollouts, users can automatically detect performance degradation or errors that may occur when transitioning to a new model through automated metric checks (Metric Gates), providing a mechanism to safely roll back.<\/p>\n

In demonstrations by the presenters, when conducting a Canary Rollout from Qwen2.5-7B to Qwen3.5-9B, the gate detected a 137% regression in p95 latency at 10% traffic. At this point, the system automatically entered the “SYSTEM_PAUSED” state, and it is reported that users could cancel the rollout and execute it in reverse to return to the original state without failing live requests.<\/p>\n

A comparison of the three strategies provided is as follows:<\/p>\n

\n\n\n\n\n\n\n\n\n\n\n
Feature<\/th>\nCanary<\/th>\nBlue-green<\/th>\nRolling<\/th>\n<\/tr>\n<\/thead>\n
Traffic Pattern<\/td>\nMaintains defined shares (default 5% \u2192 25% \u2192 50% \u2192 100%) at each step<\/td>\nBulk switch from 0% to 100%<\/td>\nSequentially replaces per replica, moving traffic according to replica ratio<\/td>\n<\/tr>\n
Additional Capacity<\/td>\nAmount close to Source size. Target grows by 1 step before Source is drained<\/td>\nBoth deployments run at full size until Source is drained<\/td>\nSource replicas per count at each step. Requires 1 additional replica during the step<\/td>\n<\/tr>\n
Typical Duration<\/td>\nInitial cold start + wait time per step (at least 390s per step when using metric gates)<\/td>\nInitial cold start + 30s propagation time. Around a few minutes<\/td>\nCold start per replica. Slower for large deployments<\/td>\n<\/tr>\n
Metric Gates<\/td>\nYes (after each step)<\/td>\nNo (due to no wait time)<\/td>\nNo<\/td>\n<\/tr>\n
How to Revert<\/td>\nFix current share by cancellation and execute rollout in reverse<\/td>\nExecute rollout in reverse. Can temporarily preserve old model with --final-source-replicas 1<\/code><\/td>\nExecute rollout in reverse<\/td>\n<\/tr>\n
Best Used For<\/td>\nWhen you want to measure on live traffic before 100% migration<\/td>\nFastest switch when short-term double capacity can be tolerated<\/td>\nWhen GPU footprint must be kept constant, or for engine\/configuration changes of the same model<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Prerequisites<\/h2>\n

The conditions for this method and feature to operate are as follows:<\/p>\n