Understand this story
Ant Group's inclusionAI has open-sourced ASpec on GitHub, a high-throughput training framework for speculative decoding draft models. The framework is built on the TorchSpec codebase, optimizes draft forward, backward, and optimizer execution, and overlaps training with target feature generation. In the Qwen3-8B/4K scenario, single-GPU Draft-only reaches 15,256.44 input tokens/s, which is 1.96× that of TorchSpec, 2.07× that of SpecForge, and 2.48× that of AngelSpec; the 8-GPU online steady-state window is 77,721.37 tokens/s, corresponding to 2.28×, 2.50×, and 2.68×. The framework supports recipes such as Qwen3-8B, Qwen3.5-9B Native MTP, Qwen3.8-27B long context, and Ling tiny/flash 3.0/3.1.
AI-generated from public reports
Developments
1 confirmed developments
- Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 6
Ant inclusionAI: New GitHub RepositoryFirst-party
Ant inclusionAI Open-Sources ASpec: A High-Throughput Training Framework for Speculative Decoding Draft Models
Ant inclusionAI has open-sourced ASpec, a high-throughput training framework for speculative decoding Draft models. Built on the TorchSpec codebase, it optimizes Draft forward, backward, and optimizer execution, and overlaps training with Target feature generation. In the Qwen3-8B/4K scenario, single-GPU Draft-only reaches 15,256.44 input tokens/s, which is 1.96× that of TorchSpec, 2.07× that of SpecForge, and 2.48× that of AngelSpec; the 8-GPU online steady-state window is 77,721.37 tokens/s, corresponding to 2.28×, 2.50×, and 2.68×. The framework supports recipes such as Qwen3-8B, Qwen3.5-9B Native MTP, Qwen3.8-27B long context, and Ling tiny/flash 3.0/3.1.