vLLM cross-instance cache image mix-up: different images share the same cache key
Hao AI Lab used the vstack agent to formally verify and discovered a cross-instance cache image mix-up issue in vLLM: when both the prefix cache and the image processing cache are disabled, each server labels images with a counter starting from 1, so two vLLM servers sharing the same external KV cache store assign the same key to different images; the second server therefore answers using the cached data of another image, and this manifests as a normal cache hit with no error. The fix is for each server to add its own random component to the label, so that labels are no longer identical across servers. The team says it has found 17 bugs (and counting); this share covers 5 of them with fix PRs.
Body
(6/7) Cross-instance mix-up: a question about one image is answered from another image's cached data.
This breaks content-based caching: two different images must never share a cache key.
Normally, vLLM identifies each image by a content hash. But when both the prefix cache and image processing cache are off, it uses a counter starting at 1 on every server. Since that label still enters an external cache store key, two vLLM servers sharing the store give different images the same key. So when the second server is asked about its own image, it can find the first server's cached data under that key and answer from that other image, with a normal cache hit and no error.
Across two servers sharing an external KV cache store, the second answered from another image's cached data. Changing image labeling fixes this: each server adds its own random part, so labels across servers no longer match.
PR: https://t.co/NmMxN1qsP5