SGLang 原生支持 Reflection Beam 并支撑其生产 API
LMSYS 宣布 SGLang 已原生支持 Reflection 的 Beam 模型,并用于支撑其生产 API。Beam 是 501B 参数、每 token 激活 23B 的 MoE 模型,原生 1M token 上下文窗口,计划以 Apache 2.0 发布,SGLang 为其提供 BF16、NVFP4 和动态 FP8 支持、DFlash2 投机解码与混合注意力前缀缓存。在 4 张 NVIDIA GB300 GPU 上以 NVFP4 运行 OpenHands 多轮智能体编码回放,并发 32 时达到每 GPU 127.6K tokens/sec、每用户 185.87 tokens/sec(p50)。

d HiCache to enable cost-efficient, production-ready serving for agentic workloads. TPU recipes for Beam and access to SGLang-TorchTPU will be available soon. Sign up through this Google Form to receive the latest updates. Reinforcement learning in Miles Miles supports Beam through a Megatron plugin and runs rollouts on Reflection's SGLang build. The training backend implements Beam's sandwich normalization with depth-scaled residuals, SSMax attention scaling, hybrid sliding-window attention, ga
匹配字段正文
