Hao AI Lab 用 vstack 在 vLLM 发现流式会话越界写入共享缓存块
(4/7) Silent out-of-bounds write: a long streaming session overwrites a shared cache block.
Hao AI Lab 在 vstack 智能体形式化验证系列第 4 条中指出,vLLM 处理流式输入时只检查每个新分块的长度,不检查整个会话的长度,因此会话可能超出模型最大长度。vLLM 没有为这些额外 token 预留内存,其数据会溢出到会话内存之外,并可能落入共享缓存块,后续复用该块的请求会得到不同输出且不报错。限制 worker 写入、丢弃超出上限的 token 数据可防止该问题,相关 PR 已提交。
正文
(4/7) Silent out-of-bounds write: a long streaming session overwrites a shared cache block.
This breaks memory isolation: a request must never write outside its own memory.
With streaming input, vLLM checks each new chunk's length but not the whole session's, so a session can exceed the model's maximum length. vLLM reserves no memory for those extra tokens, so their data spills outside the session's memory and can land in a shared cache block. A later request reusing that block can get different output, with no error.
A long streaming session past the model's length limit overwrote a shared prefix cache block. Bounding the worker write prevents that, dropping token data past the limit.
PR: https://t.co/vjh9CHLCqh