一个模型搞定所有语音识别模式，省资源还更准。
All-in-One ASR: Unifying Encoder-Decoder Models of CTC, Attention, and Transducer in Dual-Mode ASR
- 用统一架构融合CTC、注意力和转换器三种识别模式
- 模型体积大幅减小，识别准确率不降反升
- 适合需要多模式支持的语音系统开发
本文提出一种统一框架All-in-One ASR，使单一模型可同时支持连接时序分类（CTC）、基于注意力的编码器-解码器（AED）以及转换器（Transducer）三种语音识别范式，并在离线与流式两种模式下运行。尽管每种架构各有优势与权衡，但独立维护多个模型带来高昂的开发与部署成本。为此，我们设计了多模式联合模块，实现不同识别模式在单模型内的无缝集成。实验表明，All-in-One ASR显著减少整体模型体积，同时在识别性能上达到甚至超越各自优化的独立模型。此外，联合解码充分利用各模式互补优势，进一步提升识别准确率。
原文摘要 · Abstract (English)
This paper proposes a unified framework, All-in-One ASR, that allows a single model to support multiple automatic speech recognition (ASR) paradigms, including connectionist temporal classification (CTC), attention-based encoder-decoder (AED), and Transducer, in both offline and streaming modes. While each ASR architecture offers distinct advantages and trade-offs depending on the application, maintaining separate models for each scenario incurs substantial development and deployment costs. To address this issue, we introduce a multi-mode joiner that enables seamless integration of various ASR modes within a single unified model. Experiments show that All-in-One ASR significantly reduces the total model footprint while matching or even surpassing the recognition performance of individually optimized ASR models. Furthermore, joint decoding leverages the complementary strengths of different ASR modes, yielding additional improvements in recognition accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。