Understand this story
The RadixArk SGLang team has open-sourced SGLang-TorchTPU. According to information published on the LMSYS Blog on October 20, 2026, the team, together with Google and Meta, launched this PyTorch-based native backend, bringing SGLang's full serving stack to Google Cloud TPU, and it has already been used in Meta's large-scale production environment. Initial support covers models including Muse Spark Open, GLM-5.3, Kimi K3, DeepSeek V4, Qwen3.8, and Gemma 4, spanning features such as speculative decoding, KV cache offloading, PD disaggregation, and FP8 KV cache. The team also used TorchTitan and Miles to complete GRPO training of Qwen3-4B on a single 4-chip TPU v7x, with the reward curve consistent with a 6-card H200 baseline within the range of random fluctuation. There are currently no subsequent development facts.
AI-generated from public reports
Latest development · Report
SGLang-TorchTPU Open-Sourced: Bringing the SGLang Serving Stack to TPUDevelopments
1 confirmed developments
- Source:LMSYS Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 20
LMSYS BlogFirst-partySelected
SGLang-TorchTPU Open-Sourced: Bringing the SGLang Serving Stack to TPU
The SGLang team, in collaboration with Google and Meta, has open-sourced SGLang-TorchTPU, a PyTorch-based native backend that brings the full SGLang serving stack to Google Cloud TPU and is already in large-scale production use at Meta. Initial support covers models including Muse Spark Open, GLM-5.3, Kimi K3, DeepSeek V4, Qwen3.8, and Gemma 4, with features such as speculative decoding, KV cache offloading, PD disaggregation, and FP8 KV cache. The team also used TorchTitan and Miles to complete GRPO training of Qwen3-4B on a single 4-chip TPU v7x, with the reward curve matching a 6× H200 baseline within random fluctuation.