AI News

Hao AI Lab uses vstack to perform agentic formal verification on vLLM, finding 17 bugs

UCSD Hao AI LabUpdated Nov 14 03:29

AI summary

Hao AI Lab, in collaboration with the vLLM project, used vstack to perform agentic formal verification on vLLM and has found 17 bugs (and counting), ranging from silent incorrect outputs to server lockups. The team disclosed the root causes and fix PRs for 5 of these bugs, for example, hybrid models can continue generating from all-zero cache blocks, and failed requests under the queue limit do not return their slots, causing the server to reject all new requests. The relevant fixes are being submitted upstream.

Why read this

SelectedAI score 66

The original article uses five specific bugs to illustrate that formal verification can uncover edge-case combination problems that manual review struggles to cover, and provides the corresponding fix PRs.

Topics

Tags

Body

(1/7) Agentic Formal Verification on vLLM 🚀

This week, we introduced vstack for agentic formal verification in AI infra; today, we're working with @vllm_project , putting vstack to test: although vLLM is already very mature, some bugs only surface in rare combinations that are hard for human reviewers to catch.

With vstack's agentic formal verification, we identified 17 (and growing!) bugs, from silent wrong output to server lockout. Below we share 5 of them, each with a fix PR ⬇️

🔌 vstack plugin: https://t.co/GZJebbjdpL 📋 Bugs Tracker: https://t.co/6f6WmtRUWU

Source

UCSD Hao AI Lab