火红语音模型突破中文语音识别性能与效率瓶颈，支持端到端交互。
FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration
- 采用编码器-适配器-大模型框架，融合大语言模型能力提升语音理解。
- 83亿参数模型在中文基准上达3.05%字符错误率，相对降低8.4%。
- 轻量版11亿参数模型兼顾性能与效率，适合资源受限场景使用。
我们提出FireRedASR，一套大规模中文自动语音识别（ASR）模型家族，旨在满足不同应用场景对高性能与高效率的需求。该系列包含两个版本：FireRedASR-LLM（8.3B参数）采用编码器-适配器-大语言模型（LLM）框架，实现端到端语音交互；在公开中文基准上平均字符错误率（CER）为3.05%，相比最新SOTA的3.33%相对降低8.4%（CERR），在视频、直播及智能助手等多源中文场景中表现优异，相对工业级基线提升24%-40%。FireRedASR-AED（1.1B参数）基于注意力编码器-解码器（AED）架构，在保持高性能的同时更紧凑，平均CER为3.18%，优于超过120亿参数的最新SOTA模型。两模型在中文方言、英语语音及歌词识别任务中均表现良好。代码与模型已开源于https://github.com/FireRedTeam/FireRedASR。
原文摘要 · Abstract (English)
We present FireRedASR, a family of large-scale automatic speech recognition (ASR) models for Mandarin, designed to meet diverse requirements in superior performance and optimal efficiency across various applications. FireRedASR comprises two variants: FireRedASR-LLM: Designed to achieve state-of-the-art (SOTA) performance and to enable seamless end-to-end speech interaction. It adopts an Encoder-Adapter-LLM framework leveraging large language model (LLM) capabilities. On public Mandarin benchmarks, FireRedASR-LLM (8.3B parameters) achieves an average Character Error Rate (CER) of 3.05%, surpassing the latest SOTA of 3.33% with an 8.4% relative CER reduction (CERR). It demonstrates superior generalization capability over industrial-grade baselines, achieving 24%-40% CERR in multi-source Mandarin ASR scenarios such as video, live, and intelligent assistant. FireRedASR-AED: Designed to balance high performance and computational efficiency and to serve as an effective speech representation module in LLM-based speech models. It utilizes an Attention-based Encoder-Decoder (AED) architecture. On public Mandarin benchmarks, FireRedASR-AED (1.1B parameters) achieves an average CER of 3.18%, slightly worse than FireRedASR-LLM but still outperforming the latest SOTA model with over 12B parameters. It offers a more compact size, making it suitable for resource-constrained applications. Moreover, both models exhibit competitive results on Chinese dialects and English speech benchmarks and excel in singing lyrics recognition. To advance research in speech processing, we release our models and inference code at https://github.com/FireRedTeam/FireRedASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。