AI News

Hao AI Lab 推出 vstack 与 veriserve 验证推理引擎

UCSD Hao AI Lab更新于 11月12日 03:45

In PD-disaggregated serving, a GPUDirect RDMA race often returns HTTP 200 OK while silently mixing one user's KV cache into another's response. Applying vstack

AI 导读

Hao AI Lab 发布面向编码智能体的 5 阶段形式化验证流程 vstack 和可验证 PD 分离推理引擎 veriserve,在生产引擎中发现 3 类 RDMA 与缓存竞态问题,并提出 6 项上游修复。veriserve 的 KV 页生命周期设计由 226 模块、516,973 条义务的 TLAPS 安全证明支撑,在 4 卡 GB200 上运行 Qwen3-32B 时平均 p50 TPOT 开销仅 +0.18%。

正文

In PD-disaggregated serving, a GPUDirect RDMA race often returns HTTP 200 OK while silently mixing one user's KV cache into another's response. Applying vstack uncovered 3 classes of RDMA and cache lifecycle bugs in production engines, resulting in 6 proposed upstream fixes.

With TLAPS and Z3, we also proved why local timeouts fail under one-sided RDMA: without hardware stop or revocation evidence, any local-only timeout policy must either risk memory corruption if it frees a page early, or hold the exposed page indefinitely.

来源

UCSD Hao AI Lab