腾讯混元 HPC-Ops 的 Attention、Router GEMM 与 MoE 算子并入 SGLang 主分支
腾讯混元开源算子库 HPC-Ops 的 Attention、Router GEMM 和 MoE 三个算子已并入 SGLang 主分支,面向 NVIDIA Hopper(SM90)GPU,并已用 Qwen3、Hy3 和 LongCat 工作负载验证。在 8× H20 上以 Hy3-FP8 同时启用 Attention 与 MoE,batch size 4–64 时 TPOT 降低 15.1–48.8%,batch size 4–16 时 TTFT 降低 3.3–6.0%;LongCat-Flash-Lite-FP8 启用 Router GEMM 后输入吞吐提升 5.5–6.1%。这些算子已在腾讯大规模生产服务中部署,其中 Dynamic Attention 与 Fused MoE 使 Hy3 模型的 TPOT 最多降低 48.8%。
EMM produces the scores used for top-k selection, where small numerical changes can alter expert choices. The selected experts then process small and uneven token groups, allowing metadata construction, token movement, intermediate storage, and launch overhead to rival the expert GEMMs themselves. HPC-Ops addresses each stage with a dedicated operator: workload-aware scheduling for Attention, a precision-aware formulation for Router GEMM, and a fused pipeline for MoE that eliminates the standalo
匹配字段正文






