Hao AI Lab Open-Sources JetSpec++ Models, Code, and Inference Engine
Hao AI Lab announced that JetSpec++'s models, code, and inference engine are all open-sourced, along with links to the paper, code, and Hugging Face. The method pipelines draft generation and verification, achieving up to 1.54x end-to-end speedup over DSpark and 7.45x over autoregressive decoding on Qwen3-8B, reaching 1,361 TPS in BF16; with NVFP4 and kernel optimizations, it exceeds 4,000 TPS on two GB200s.
Why read this
AI score 59
The original article provides the complete open-source entry points for JetSpec++ and a round-by-round latency comparison with DSpark, allowing readers to assess the engineering benefits of speculative decoding.
Topics
Body
[5/5] JetSpec++ models, code and inference engine are fully open-sourced. Please give it a try and let us know what you think in the comments!
Check out our paper and the link below for more details!
Paper: https://t.co/GY4W71d7LI Code: https://t.co/Oj92K5fwAR HF: https://t.co/tSXtfsLUmd