AI News

Hao AI Lab launches vstack and veriserve verified inference engine

UCSD Hao AI LabUpdated Nov 12 03:45

AI summary

Hao AI Lab has released vstack, a 5-stage formal verification pipeline for coding agents, and veriserve, a verifiable PD-disaggregated inference engine. They found 3 classes of RDMA and cache race conditions in production engines and proposed 6 upstream fixes. veriserve's KV page lifecycle design is backed by a TLAPS safety proof comprising 226 modules and 516,973 obligations, and when running Qwen3-32B on 4-GPU GB200, the average p50 TPOT overhead is only +0.18%.

Body

In PD-disaggregated serving, a GPUDirect RDMA race often returns HTTP 200 OK while silently mixing one user's KV cache into another's response. Applying vstack uncovered 3 classes of RDMA and cache lifecycle bugs in production engines, resulting in 6 proposed upstream fixes.

With TLAPS and Z3, we also proved why local timeouts fail under one-sided RDMA: without hardware stop or revocation evidence, any local-only timeout policy must either risk memory corruption if it frees a page early, or hold the exposed page indefinitely.

Source

UCSD Hao AI Lab