Understand this story
On October 21, 2026, LMSYS published a blog post introducing Elastic EP Scaling (elastic expert parallelism scaling) in SGLang. This mechanism allows MoE deployments to add GPUs in fine-grained increments without requiring a full replica or restart; in the example, it scales from 20 GPUs to 24 GPUs. At startup, max_ep_size reserves metadata and communication buffer capacity, and new ranks start on demand. Tested with DeepSeek-V3-Lite FP8 on 8×H100, the first completed request was 5.3×–8.2× faster than with a restart. LMSYS also noted that after a rank joins the Mooncake WORLD, automatic rollback is not currently supported, and a failure during finalization may require restarting the deployment. There are currently no subsequent developments.
AI-generated from public reports
Latest development · Report
SGLang's Elastic Expert Parallelism Scaling: Online Scaling of MoE Deployments with NIXLDevelopments
1 confirmed developments
- Source:LMSYS Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 21
LMSYS BlogFirst-party
SGLang's Elastic Expert Parallelism Scaling: Online Scaling of MoE Deployments with NIXL
SGLang's Elastic EP Scaling lets MoE deployments add GPUs in fine-grained increments without full replicas or restarts, scaling from 20 GPUs to 24 GPUs in the example. max_ep_size reserves metadata and communication buffer capacity at startup, and new ranks start on demand; tested on 8×H100 with DeepSeek-V3-Lite FP8, the first completed request was 5.3×–8.2× faster than a restart. After a joining rank enters the Mooncake WORLD, automatic rollback is not currently supported, and a failure during finalization may require restarting the deployment.