AI News

Hao AI Lab 用 vstack 在 vLLM 发现 17 个 bug

UCSD Hao AI Lab更新于 11月14日 03:29

(2/7) Silent wrong output: hybrid models can continue from an all-zero cache block, and then cache it.

AI 导读

Hao AI Lab 用 vstack 对 vLLM 做智能体形式化验证,已发现 17 个(还在增加)bug,并公开其中 5 个及修复 PR。混合模型开启细粒度前缀匹配时,vLLM 在缓存块写入前就标记可复用,同一步的第二个请求会复制到全零块并继续生成,GB200 上修复前 32/32 次复现全零、修复后为 0。其余问题包括队列上限下 2 个失败请求即可让服务持续返回 busy、流式会话越界写共享缓存块、LoRA 原地重载后新请求仍用旧权重、跨实例图像缓存键冲突。

正文

(2/7) Silent wrong output: hybrid models can continue from an all-zero cache block, and then cache it.

This breaks prefix caching: a cache hit must give a request the same result that it would have computed itself.

On hybrid models with fine-grained prefix matching turned on, vLLM marks a cache block as reusable as soon as it plans the block, before the block is actually written. So when a second request in the same step reuses that block, it copies it too early and gets all zeros. The second request then continues from those zeros. Since the bad block stays in the cache, later requests can pick it up as well, while every response still looks normal.

On GB200, the copy source was all zeros in 32 of 32 trials before the fix, and 0 after. A small fix defers a request with an unwritten copy source to the next scheduler step, after the source is populated.

PR: https://t.co/NMPdqdCAxv

来源

UCSD Hao AI Lab