AI News

UCSD Hao AI Lab launches vstack and veriserve verifiable inference engine

UCSD Hao AI LabUpdated Nov 12 03:46

AI summary

UCSD Hao AI Lab releases vstack, a five-stage formal verification workflow, and veriserve, a verifiable PD-disaggregated inference engine. It discovered 3 classes of real RDMA and cache races in production engines and proposed 6 upstream fixes. veriserve's KV-page lifecycle design model is backed by a TLAPS safety proof with 226 modules and 516,973 obligations, and when running Qwen3-32B on a 4-GPU GB200, the average p50 TPOT overhead is only +0.18%.

Body

Towards Verifiable LLM Infra 🚀

Suppose coding agents can build a fast inference engine from scratch, and one engineer orchestrates 10,000 agents in parallel: how do we know the engine they wrote is actually correct? Having agents review other agents mostly wastes tokens compared to empirical and formal verification.

We introduce vstack, a 5-stage formal verification workflow for coding agents, and veriserve, a verifiable PD-disaggregated serving engine: uncovering 3 classes of real RDMA and cache races in production serving engines, and backing veriserve's KV-page lifecycle design model with a 226-module, 516,973-obligation TLAPS safety proof at +0.18% mean p50 TPOT overhead on GB200.

Blog: https://t.co/SdbINvSVht veriserve: https://t.co/ugHLToReCN vstack: https://t.co/GZJebbjdpL

Source

UCSD Hao AI Lab