Understand this story
Hao AI Lab used the vstack agent-based formal verification tool to test vLLM and discovered a cross-instance cache graph-mixing issue in vLLM. The team explained that when both the prefix cache and the image processing cache are disabled, each server labels images with a count starting from 1; two vLLM servers sharing an external KV cache store will assign the same key to different images, so the second server answers using the cached data of another image, and this manifests as a normal cache hit with no error. The fix is for each server to add its own random component to the label, so that labels are no longer identical across servers. The team said it has found 17 bugs (and counting), and this sharing covers 5 of them along with fix PRs.
AI-generated from public reports
Latest development · Report
vLLM cross-instance cache image mix-up: different images share the same cache keyDevelopments
1 confirmed developments
Hao AI Lab uses vstack to perform agentic formal verification on vLLM
Source:UCSD Hao AI Lab
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 14
UCSD Hao AI LabFirst-party
vLLM cross-instance cache image mix-up: different images share the same cache key
Hao AI Lab used the vstack agent to formally verify and discovered a cross-instance cache image mix-up issue in vLLM: when both the prefix cache and the image processing cache are disabled, each server labels images with a counter starting from 1, so two vLLM servers sharing the same external KV cache store assign the same key to different images; the second server therefore answers using the cached data of another image, and this manifests as a normal cache hit with no error. The fix is for each server to add its own random component to the label, so that labels are no longer identical across servers. The team says it has found 17 bugs (and counting); this share covers 5 of them with fix PRs.