Hao AI Lab 推出 veriserve 可验证推理引擎
veriserve resolves this trade-off with PdLedger, tracking every physical page with a versioned (page, gen) ticket. Timed-out pages without stop evidence stay is
UCSD Hao AI Lab 发布 veriserve 可验证 PD 分离推理引擎,用 PdLedger 以带版本的 (page, gen) 票据追踪每个物理页,超时且无停止证据的页面留在 QUARANTINED 隔离,避免污染 KV cache。在 4 卡 NVIDIA GB200、Qwen3-32B 上,PdLedger 仅增加 0.18% 的 p50 TPOT 均值开销(95% CI 上界 +0.68%),拦截全部 3 个注入的 glue 故障,并通过 TLC 回放 100% 的 22,104 个边界事件。配套 vstack 五阶段形式化验证流程还发现生产引擎中 3 类 RDMA 与 cache 竞态,提出 6 项上游修复。
正文
veriserve resolves this trade-off with PdLedger, tracking every physical page with a versioned (page, gen) ticket. Timed-out pages without stop evidence stay isolated in QUARANTINED so requests fail fast without corrupting the cache, while completed transfers drain cleanly to FREE.
On a 4-GPU NVIDIA GB200 node with Qwen3-32B, PdLedger adds only +0.18% mean p50 TPOT overhead with a +0.68% 95% CI upper bound, rejects all 3 injected glue faults, and replays 100% of 22,104 boundary events through TLC. More verifiable serving is coming soon!