vLLM 跨实例缓存串图:不同图片共用同一缓存键
(6/7) Cross-instance mix-up: a question about one image is answered from another image's cached data.
Hao AI Lab 用 vstack 智能体形式化验证在 vLLM 中发现跨实例缓存串图问题:当 prefix cache 与图像处理缓存均关闭时,每个服务器从 1 开始计数给图片打标签,两个共享外部 KV cache 存储的 vLLM 服务器会给不同图片相同键,第二台服务器因此用另一张图片的缓存数据作答,且表现为正常缓存命中、不报错。修复方式是每台服务器在标签中加入自己的随机部分,使跨服务器标签不再相同。该团队称共发现 17 个(仍在增加)bug,本次分享其中 5 个并附修复 PR。
正文
(6/7) Cross-instance mix-up: a question about one image is answered from another image's cached data.
This breaks content-based caching: two different images must never share a cache key.
Normally, vLLM identifies each image by a content hash. But when both the prefix cache and image processing cache are off, it uses a counter starting at 1 on every server. Since that label still enters an external cache store key, two vLLM servers sharing the store give different images the same key. So when the second server is asked about its own image, it can find the first server's cached data under that key and answer from that other image, with a normal cache hit and no error.
Across two servers sharing an external KV cache store, the second answered from another image's cached data. Changing image labeling fixes this: each server adds its own random part, so labels across servers no longer match.
PR: https://t.co/NmMxN1qsP5