Hao AI Lab 用 vstack 发现 SGLang 取消请求丢失问题
(6/8) Lost cancellation: under DP attention, a rapid cancel can overtake its own generation request and be dropped, leaving the request running to max_new_token
Hao AI Lab 用 vstack 在 SGLang 中发现 DP attention 下的取消丢失问题:快速 abort 可能先于其生成请求到达而被丢弃,请求继续跑到 max_new_tokens。在 GB200 上,20 ms 内发出的 204 次 abort 中有 33 次丢失并跑满 max_new_tokens;改为在每个 rank 的 work socket 上按序传输 abort 后降为 0/204。该问题已提交修复 PR。
正文
(6/8) Lost cancellation: under DP attention, a rapid cancel can overtake its own generation request and be dropped, leaving the request running to max_new_tokens.
This breaks cancellation finality: once an abort request succeeds, the cancelled request must not run to completion.
With the default DP control routing, a generation request goes directly to its owning rank while an abort goes to rank 0 and is broadcast across ranks, with no ordering guarantee between the two paths. If the abort arrives first, the scheduler has not seen the request yet, drops the abort, and runs the request to its token limit on the next poll.
On GB200, 33 of 204 aborts sent within 20 ms were lost and ran to max_new_tokens, dropping to 0 of 204 once aborts travel in order on each rank's work socket.
PR: https://t.co/wI4nH8vUq7
AI 热点