Hao AI Lab 用 vstack 发现 SGLang 解码服务器卡死 364 秒
(3/8) Decode server stall: on a PD-disaggregated decode server, a single retracted request freezes the transfer queue while health checks still pass.
Hao AI Lab 用 vstack 在 SGLang 上发现 PD 分离解码服务器卡死问题:单个被撤回请求会冻结传输队列,而 /health 仍返回 200。在 GB200 上解码服务器卡顿 364.69 秒,363 次健康检查全部通过;只对新预分配加门控即可让已到达的传输完成并解除卡顿。该问题属于其与 SGLang 社区合作提出的 20+ 个问题与修复 PR 之一。
正文
(3/8) Decode server stall: on a PD-disaggregated decode server, a single retracted request freezes the transfer queue while health checks still pass.
This breaks liveness: requests whose KV cache has already arrived must be able to run and release their reserved memory so retracted work can resume.
Under KV pressure, the decode scheduler retracts a request and returns early every step to hold off new preallocations. That early return also skips polling the transfer queue, so arrived transfers keep holding their pages and decode reserves while the retracted request never fits, even as /health still returns 200.
On GB200, the decode server stalled for 364.69 s while /health answered 200 in 363 of 363 probes. Gating only new preallocations lets arrived transfers finish and clears the stall.
PR: https://t.co/55VVQdpLSY
AI 热点