Hao AI Lab uses vstack to perform agentic formal verification on vLLM, finding 17 bugs
Hao AI Lab, in collaboration with the vLLM project, used vstack to perform agentic formal verification on vLLM and has found 17 bugs (and counting), ranging from silent incorrect outputs to server lockups. The team disclosed the root causes and fix PRs for 5 of these bugs, for example, hybrid models can continue generating from all-zero cache blocks, and failed requests under the queue limit do not return their slots, causing the server to reject all new requests. The relevant fixes are being submitted upstream.
Why read this
SelectedAI score 66
The original article uses five specific bugs to illustrate that formal verification can uncover edge-case combination problems that manual review struggles to cover, and provides the corresponding fix PRs.
Topics
Body
(1/7) Agentic Formal Verification on vLLM 🚀
This week, we introduced vstack for agentic formal verification in AI infra; today, we're working with @vllm_project , putting vstack to test: although vLLM is already very mature, some bugs only surface in rare combinations that are hard for human reviewers to catch.
With vstack's agentic formal verification, we identified 17 (and growing!) bugs, from silent wrong output to server lockout. Below we share 5 of them, each with a fix PR ⬇️
🔌 vstack plugin: https://t.co/GZJebbjdpL 📋 Bugs Tracker: https://t.co/6f6WmtRUWU