FireRedTeam 开源一体化语音系统 FireRedASR2S
GitHub - FireRedTeam/FireRedASR2S: A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en. · GitHub
FireRedTeam 开源 FireRedASR2S，一套包含 ASR、VAD、LID、Punc 四个模块的一体化语音系统，各模块均称达到 SOTA。FireRedASR2 支持中文（普通话及 20 多种方言口音）、英文、中英混说以及语音和歌声转写，在 4 个普通话公开基准上平均 CER 为 2.89%，在 19 个中文方言口音基准上为 11.55%，均优于 Doubao-ASR、Qwen3-ASR-1.7B、Fun-ASR 和 Fun-ASR-Nano-2512。FireRedVAD 支持 100 多种语言的语音、歌声和音乐检测，F1 为 97.57%；FireRedLID 覆盖 100 多种语言和 20 多种中文方言，准确率 97.18%；FireRedPunc 中英文标点预测平均 F1 为 78.90%。四个模块可独立使用，模型权重与推理代码已放出，技术报告见 arXiv。
推荐理由
精选AI 评分 60
FireRedASR2S 把 ASR、VAD、LID、Punc 四个模块打包开源并给出各基准对比数据，可据此判断中文方言与歌声转写的可用性。
主题
正文
当前显示原文，中文全文译文尚未就绪。
FireRedTeam; FireRedASR2S
Repository navigation
NotificationsYou must be signed in to change notification settings; Fork47 (47); Star714 (714)You must be signed in to star a repository
A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en.
Repository files navigation
README; Apache-2.0 license
[Paper] [Model🤗] [Model🤖] [Demo]
FireRedASR2S is a state-of-the-art (SOTA), industrial-grade, all-in-one ASR system with ASR, VAD, LID, and Punc modules. All modules achieve SOTA performance:
FireRedASR2: Automatic Speech Recognition (ASR) supporting speech and singing transcription for Chinese (Mandarin, 20+ dialects/accents), English, code-switching. 2.89% average CER on 4 public Mandarin benchmarks, 11.55% on 19 Chinese dialects and accents benchmarks, outperforming Doubao-ASR, Qwen3-ASR-1.7B, Fun-ASR, and Fun-ASR-Nano-2512. FireRedASR2-AED also supports word-level timestamps and confidence scores; FireRedVAD: Voice Activity Detection (VAD) supporting speech/singing/music in 100+ languages. 97.57% F1, outperforming Silero-VAD, TEN-VAD, FunASR-VAD and WebRTC-VAD. Supports non-streaming/streaming VAD and Multi-label VAD (mVAD); FireRedLID: Spoken Language Identification (LID) supporting 100+ languages and 20+ Chinese dialects/accents. 97.18% accuracy, outperforming Whisper and SpeechBrain; FireRedPunc: Punctuation Prediction (Punc) for Chinese and English. 78.90% average F1, outperforming FunASR-Punc (62.77%)
2S: 2nd-generation FireRedASR, now expanded to an all-in-one ASR System
[2026.03.12] 🔥 We release FireRedASR2S technical report. See arXiv; [2026.03.05] 🚀 vLLM supports FireRedASR2-LLM. See vLLM Usage part; [2026.02.25] 🔥 We release FireRedASR2-LLM model weights. 🤗 🤖; [2026.02.13] 🚀 Support TensorRT-LLM inference acceleration for FireRedASR2-AED (contributed by NVIDIA). Benchmark on AISHELL-1 test set shows 12.7x speedup over PyTorch baseline (single H20); [2026.02.12] 🔥 We release FireRedASR2S (FireRedASR2-AED, FireRedVAD, FireRedLID, and FireRedPunc) with model weights and inference code. Download links below. Technical report and finetuning code coming soon
Available Models and Languages
*Supported Chinese dialects/accents: Cantonese (Hong Kong & Guangdong), Sichuan, Shanghai, Wu, Minnan, Anhui, Fujian, Gansu, Guizhou, Hebei, Henan, Hubei, Hunan, Jiangxi, Liaoning, Ningxia, Shaanxi, Shanxi, Shandong, Tianjin, Yunnan, etc.
Evaluation
FireRedASR2
Metrics: Character Error Rate (CER%) for Chinese and Word Error Rate (WER%) for English. Lower is better.
We evaluate FireRedASR2 on 24 public test sets covering Mandarin, 20+ Chinese dialects/accents, and singing.
Mandarin (4 test sets): 2.89% (LLM) / 3.05% (AED) average CER, outperforming Doubao-ASR (3.69%), Qwen3-ASR-1.7B (3.76%), Fun-ASR (4.16%) and Fun-ASR-Nano-2512 (4.55%); Dialects (19 test sets): 11.55% (LLM) / 11.67% (AED) average CER, outperforming Doubao-ASR (15.39%), Qwen3-ASR-1.7B (11.85%), Fun-ASR (12.76%) and Fun-ASR-Nano-2512 (15.07%)
Note: FRASR2=FireRedASR2, ws=WenetSpeech, md=MagicData, conv=Conversational, daily=Daily-use.
FireRedVAD
FireRedVAD achieves SOTA performance, outperforming Silero-VAD, TEN-VAD, FunASR-VAD, and WebRTC-VAD.
FLEURS-VAD-102: We randomly selected ~100 audio files per language from FLEURS test set, resulting in 9,443 audio files with manually annotated binary VAD labels (speech=1, silence=0). This VAD testset will be open sourced (coming soon).
Note: FunASR-VAD achieves low Miss Rate but at the cost of high False Alarm Rate (44.03%), indicating over-prediction of speech segments.
FireRedLID
We evaluate FireRedLID on multilingual and Chinese dialect benchmarks.
FireRedLID achieves SOTA performance, outperforming Whisper, SpeechBrain-LID, and Dolphin.
FireRedPunc
We evaluate FireRedPunc on multi-domain Chinese and English benchmarks.
FireRedPunc achieves SOTA performance, outperforming FunASR-Punc (CT-Transformer).
Method
FireRedASR2S: System Overview
FireRedASR2
FireRedASR2 builds upon FireRedASR with improved accuracy, designed to meet diverse requirements in superior performance and optimal efficiency across various applications. It comprises two variants:
FireRedASR2-LLM: Designed to achieve state-of-the-art performance and to enable seamless end-to-end speech interaction. It adopts an Encoder-Adapter-LLM framework leveraging large language model (LLM) capabilities; FireRedASR2-AED: Designed to balance high performance and computational efficiency and to serve as an effective speech representation module in LLM-based speech models. It utilizes an Attention-based Encoder-Decoder (AED) architecture
Other Modules
FireRedVAD: DFSMN-based non-streaming/streaming Voice Activity Detection and Multi-label VAD (mVAD). mVAD can be viewed as a lightweight Audio Event Detection (AED) system specialized for a small set of sound categories (speech/singing/music); FireRedLID: Encoder-Decoder-based Spoken Language Identification. See FireRedLID README for language details; FireRedPunc: BERT-based Punctuation Prediction
Quick Start
Setup
Create a clean Python environment:
Install dependencies and set up PATH and PYTHONPATH:
Download models:
Convert your audio to 16kHz 16-bit mono PCM format if needed:
The four components under fireredasr2s, i.e. fireredasr2, fireredvad, fireredlid, and fireredpunc are self-contained and designed to work as a standalone modules. You can use any of them independently without depending on the others. FireRedVAD and FireRedLID will also be open-sourced as standalone libraries in separate repositories.
Set up PYTHONPATH first: export PYTHONPATH=$PWD/:$PYTHONPATH
mVAD (Audio Event Detection, AED)
Note: FireRedASR2S code has only been tested on Linux Ubuntu 22.04. Behavior on other Linux distributions or Windows has not been tested.
FAQ
Q: What audio format is supported?
16kHz 16-bit mono PCM wav. Use ffmpeg to convert other formats: ffmpeg -i -ar 16000 -ac 1 -acodec pcm_s16le -f wav
Q: What are the input length limitations of ASR models?
FireRedASR2-AED supports audio input up to 60s. Input longer than 60s may cause hallucination issues, and input exceeding 200s will trigger positional encoding errors; FireRedASR2-LLM supports audio input up to 40s. The behavior for longer input is untested; FireRedASR2-LLM Batch Beam Search: When performing batch beam search with FireRedASR2-LLM, even though attention masks are applied, it is recommended to ensure that the input lengths of the utterances are similar. If there are significant differences in utterance lengths, shorter utterances may experience repetition issues. You can either sort your dataset by length or set batch_size to 1 to avoid the repetition issue
Acknowledgements
Thanks to the following open-source works:
Qwen; WenetSpeech-Yue; WenetSpeech-Chuan
Citation
A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en.