AI 热点

Hao AI Lab 用 vstack 修复 SGLang 多 GPU 通信错位

UCSD Hao AI Lab更新于 11月18日 02:32

(5/8) Multi-GPU rank mismatch: under DP attention with the prefill delayer and interval min batch size, two DP ranks enter different collectives and exit.

AI 导读

Hao AI Lab 用 vstack 在 SGLang 上定位并修复了 DP attention 下 prefill delayer 与 interval min batch size 引发的多 GPU rank 错位:忙 rank 跳过 prefill 直接进入每轮同步,空闲 rank 却进入 delayer 协商,通信库随即中止。让延迟 prefill 的 rank 以零 prefill 仍参与协商后,GB200 上 30 次跨阈值运行全部完成、无中止。

正文

(5/8) Multi-GPU rank mismatch: under DP attention with the prefill delayer and interval min batch size, two DP ranks enter different collectives and exit.

This breaks multi-GPU lockstep: every rank sharing a communication group must run the exact same collectives in the exact same order each step.

Each rank checks its own running batch size when deciding whether to defer prefill. When one rank meets the threshold and another sits below it, the busy rank skips prefill and goes straight to the per-round sync while the idle rank enters the prefill-delayer negotiation on the same CPU group, causing the communication library to abort.

Having a deferring rank still join the delayer negotiation with zero prefill keeps every collective aligned, completing 30 of 30 cross-threshold runs on GB200 with no abort.

PR: https://t.co/lhGKES67Cw

来源

UCSD Hao AI Lab