AI News

Hao AI Lab launches veriserve, a verifiable inference engine

UCSD Hao AI LabUpdated Nov 12 03:45

AI summary

UCSD Hao AI Lab releases veriserve, a verifiable PD-disaggregated inference engine, which uses PdLedger to track each physical page with versioned (page, gen) tickets; pages that time out without evidence of stopping remain in QUARANTINED isolation, avoiding contamination of the KV cache. On 4-GPU NVIDIA GB200 with Qwen3-32B, PdLedger adds only 0.18% mean p50 TPOT overhead (95% CI upper bound +0.68%), intercepts all 3 injected glue faults, and replays 100% of 22,104 boundary events via TLC. The accompanying vstack five-stage formal verification process also found 3 classes of RDMA and cache races in the production engine and proposed 6 upstream fixes.

Body

veriserve resolves this trade-off with PdLedger, tracking every physical page with a versioned (page, gen) ticket. Timed-out pages without stop evidence stay isolated in QUARANTINED so requests fail fast without corrupting the cache, while completed transfers drain cleanly to FREE.

On a 4-GPU NVIDIA GB200 node with Qwen3-32B, PdLedger adds only +0.18% mean p50 TPOT overhead with a +0.68% 95% CI upper bound, rejects all 3 injected glue faults, and replays 100% of 22,104 boundary events through TLC. More verifiable serving is coming soon!

Source

UCSD Hao AI Lab