AI Trends

Trending story

Hao AI Lab discloses SGLang decode server stall issue

1 reports1 story updatesUpdated Nov 18 02:27

Understand this story

AI overview

Hao AI Lab discloses a stall issue in SGLang's PD-disaggregated decode servers: a single aborted request freezes the transfer queue, while the /health check still returns 200. The team observed with vstack on GB200 that the decode server stalled for 364.69 seconds, during which all 363 health checks passed. Hao AI Lab states that gating only new preallocations is enough to let already-arrived transfers complete and resolve the stall. This issue is one of 20+ issues and fix PRs the team proposed in collaboration with the SGLang community.

AI-generated from public reports

Latest development · Report

Hao AI Lab uses vstack to discover SGLang decode server hang of 364 seconds

Developments

1 confirmed developments

  1. Hao AI Lab discloses SGLang decoding server stall issue

    Source:UCSD Hao AI Lab

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 18

  1. UCSD Hao AI LabFirst-party

    Hao AI Lab uses vstack to discover SGLang decode server hang of 364 seconds

    Hao AI Lab used vstack to discover a PD-disaggregated decode server hang issue in SGLang: a single aborted request freezes the transfer queue, while /health still returns 200. On GB200, the decode server stalled for 364.69 seconds, and all 363 health checks passed; gating only new preallocations allows already-arrived transfers to complete and resolves the hang. This issue is one of 20+ issues and fix PRs proposed in collaboration with the SGLang community.

Related stories

  1. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  2. Related storiesMistral releases Large 4 weights on Hugging Face
  3. Related storiesSGLang natively supports and powers the Beam production API
  4. Related storiesMeta releases open-weight model Muse Spark Open