改进语音识别中的帧聚合方法，让中英文非自回归模型更准确
UMA-Split: unimodal aggregation for both English and Mandarin non-autoregressive speech recognition
- 用可分裂的帧聚合模块，让单帧对应多个文本词元
- 在英语上实现比传统CTC高4.3%的字错误率降低
- 适合需要快速推理的中英文语音识别场景
本文提出一种基于统一模态聚合（UMA）的非自回归语音识别模型，适用于英语和中文。原始的UMA通过单调先增后减的权重对同一文本词元的声学帧进行分段聚合，以学习更优表征，但仅在中文表现良好。在英语中，单个音节常被切分为多个细粒度词元，或词元覆盖少于3帧，导致无法形成有效的单峰权重。为解决此问题，我们引入一个简单的分裂模块，在计算CTC损失前将每个聚合帧映射到多个词元，从而提升跨语言泛化能力。实验表明，该方法在英语上达到4.3%的字错误率下降，显著优于基线。
原文摘要 · Abstract (English)
This paper proposes a unimodal aggregation (UMA) based nonautoregressive model for both English and Mandarin speech recognition. The original UMA explicitly segments and aggregates acoustic frames (with unimodal weights that first monotonically increase and then decrease) of the same text token to learn better representations than regular connectionist temporal classification (CTC). However, it only works well in Mandarin. It struggles with other languages, such as English, for which a single syllable may be tokenized into multiple fine-grained tokens, or a token spans fewer than 3 acoustic frames and fails to form unimodal weights. To address this problem, we propose allowing each UMA-aggregated frame map to multiple tokens, via a simple split module that generates two tokens from each aggregated frame before computing the CTC loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。