Hao AI Lab 开源 JetSpec++ 投机解码框架
[2/5] With JetSpec++, drafting time is almost removed from the critical path. Walltime reported below shows how latency reduces for each draft-verification roun
UCSD Hao AI Lab 推出 JetSpec++，通过因果并行草稿与验证流水线化，在 Qwen3-8B 上实现 1.54 倍于 DSpark、7.45 倍于自回归解码的无损端到端加速，BF16 下达 1,361 TPS。GSM8K 上 Qwen3-8B 每轮耗时从 DSpark 的 9.92 ms 降至 6.84 ms，草稿时间由 3.76 ms 压缩到 0.61 ms。模型、代码与推理引擎已全部开源。
正文
[2/5] With JetSpec++, drafting time is almost removed from the critical path. Walltime reported below shows how latency reduces for each draft-verification round, and its breakdown on GSM8K: • Qwen3-8B: DSpark 9.92 ms (including 3.76 ms drafting time), JetSpec++ 6.84 ms (including 0.61 ms is drafting time) • Qwen3.8-27B: DSpark 31.88 ms (4.46 ms drafting time), JetSpec++ 28.47 ms (0.71 ms drafting time)
The key is JetSpec++ has effective foresight drafting: it guesses the verification outcome correctly in 92.9% (8B) and 89.0% (27B) of rounds, so the next block is already drafted when verification ends.