Understand this story
On October 29, 2026, the Baseten engineering team published a blog post describing optimizations for deploying Qwen3-ASR 1.7B batch transcription. Through native audio decoding, FP8 quantization of the KV cache, and reduced host-device synchronization, the team increased batch transcription throughput by 7x and reduced latency by 6.9x on a single RTX PRO 6000. After optimization, transcribing one hour of audio costs about $0.0013, or 1/10 of a cent per audio hour, with no noticeable change in word error rate. The optimized version is now the default deployment in the Baseten Model Library and can be called through an OpenAI-compatible transcription client.
AI-generated from public reports
Latest development · Report
Baseten Optimizes Qwen3-ASR Batch Transcription, Boosting Single-GPU Throughput by 7xDevelopments
1 confirmed developments
- Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 29
Baseten Engineering BlogFirst-party
Baseten Optimizes Qwen3-ASR Batch Transcription, Boosting Single-GPU Throughput by 7x
The Baseten engineering team, through native audio decoding, KV cache FP8 quantization, and reduced host-device synchronization, increased batch transcription throughput for Qwen3-ASR 1.7B on a single RTX PRO 6000 by 7x and reduced latency by 6.9x. After optimization, the transcription cost for one hour of audio is approximately $0.0013, i.e., 1/10 of a cent per audio hour, with no significant change in word error rate. The optimized version has been made the default deployment in the Baseten Model Library and can be called through an OpenAI-compatible transcription client.