AI News

Trending story

Hao AI Lab uses vstack to perform agent-based formal verification of vLLM

1 reports1 story updatesUpdated Nov 14 03:13

Understand this story

AI overview

Hao AI Lab used the vstack agent-based formal verification tool to test vLLM and discovered a cross-instance cache graph-mixing issue in vLLM. The team explained that when both the prefix cache and the image processing cache are disabled, each server labels images with a count starting from 1; two vLLM servers sharing an external KV cache store will assign the same key to different images, so the second server answers using the cached data of another image, and this manifests as a normal cache hit with no error. The fix is for each server to add its own random component to the label, so that labels are no longer identical across servers. The team said it has found 17 bugs (and counting), and this sharing covers 5 of them along with fix PRs.

AI-generated from public reports

Latest development · Report

vLLM cross-instance cache image mix-up: different images share the same cache key

Developments

1 confirmed developments

  1. Hao AI Lab uses vstack to perform agentic formal verification on vLLM

    Source:UCSD Hao AI Lab

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 14

  1. UCSD Hao AI LabFirst-party

    vLLM cross-instance cache image mix-up: different images share the same cache key

    Hao AI Lab used the vstack agent to formally verify and discovered a cross-instance cache image mix-up issue in vLLM: when both the prefix cache and the image processing cache are disabled, each server labels images with a counter starting from 1, so two vLLM servers sharing the same external KV cache store assign the same key to different images; the second server therefore answers using the cached data of another image, and this manifests as a normal cache hit with no error. The fix is for each server to add its own random component to the label, so that labels are no longer identical across servers. The team says it has found 17 bugs (and counting); this share covers 5 of them with fix PRs.

Related stories

  1. Related storiesHao AI Lab uses vstack to perform agent-based formal verification of vLLM
  2. Related storiesAnthropic launches Claude Fable 5.2 and Claude Mythos 5.2
  3. Related storiesMistral releases Large 4 weights on Hugging Face
  4. Related storiesNVIDIA releases OpenShell reference architecture