AI News

UCSD Hao AI Lab 推出 vstack 与 veriserve 可验证推理引擎

UCSD Hao AI Lab更新于 11月12日 03:46

Towards Verifiable LLM Infra 🚀

AI 导读

UCSD Hao AI Lab 发布 vstack 五阶段形式化验证工作流与可验证 PD 分离式推理引擎 veriserve,在生产引擎中发现 3 类真实 RDMA 与缓存竞态,并提出 6 项上游修复。veriserve 的 KV-page 生命周期设计模型由 226 个模块、516,973 条义务的 TLAPS 安全证明支撑,在 4 卡 GB200 上运行 Qwen3-32B 时平均 p50 TPOT 开销仅 +0.18%。

正文

Towards Verifiable LLM Infra 🚀

Suppose coding agents can build a fast inference engine from scratch, and one engineer orchestrates 10,000 agents in parallel: how do we know the engine they wrote is actually correct? Having agents review other agents mostly wastes tokens compared to empirical and formal verification.

We introduce vstack, a 5-stage formal verification workflow for coding agents, and veriserve, a verifiable PD-disaggregated serving engine: uncovering 3 classes of real RDMA and cache races in production serving engines, and backing veriserve's KV-page lifecycle design model with a 226-module, 516,973-obligation TLAPS safety proof at +0.18% mean p50 TPOT overhead on GB200.

Blog: https://t.co/SdbINvSVht veriserve: https://t.co/ugHLToReCN vstack: https://t.co/GZJebbjdpL

来源

UCSD Hao AI Lab