AI 热点

Hao AI Lab 用 vstack 在 SGLang 发现 20+ 并发缺陷

UCSD Hao AI Lab更新于 11月18日 02:32

(2/8) Weight-update race condition: after an in-place pause in an RL loop, a weight update can overwrite weights while an in-flight step is still reading them.

AI 导读

Hao AI Lab 与 SGLang 社区用 vstack 在 SGLang 中找出 20+ 个并发与多步状态缺陷,并提交修复 PR。其中权重更新竞态在 GB200 上修复前 8/8 次试验中在途步骤用新权重算出 token、logprob 和 KV,修复后为 0/8;decode server 卡死 364.69 秒期间 /health 仍 363/363 返回 200。修复方式包括参数写入前加 GPU stream 等待、只限制新预分配等。

正文

(2/8) Weight-update race condition: after an in-place pause in an RL loop, a weight update can overwrite weights while an in-flight step is still reading them.

This breaks rollout policy isolation: a forward step launched before a pause must finish reading the old weights before a weight update changes them.

With the default overlap scheduler, an in-place pause returns right away while the already-launched step keeps running on the GPU's forward stream. Parameter writes run on the scheduler stream, whose barrier only waits for input metadata reads before the model layers, so new weights can land mid-step with no error.

On GB200, the in-flight step computed its token, logprob, and KV from the updated weights in 8 of 8 trials before the fix, and 0 of 8 after. Adding a GPU stream wait before parameter writes ensures the forward step finishes first.

PR: https://t.co/akyxGsdGu2

来源

UCSD Hao AI Lab