通过输入增强保持语音识别中CTC分布一致性，提升模型泛化能力。
CR-CTC: Consistency regularization on CTC for improved speech recognition
- 对同一语音的两种增强谱图分别计算CTC分布并强制一致。
- 在时间掩码区域通过掩码预测学习上下文表示，提升特征表达。
- 有效抑制尖锐输出分布，减少过拟合，适合追求高效高精度的ASR研究者。
连接时序分类（CTC）是自动语音识别中广泛应用的方法，因其简单和高效而受到青睐，但性能常有不足。本文提出一致性正则化CTC（CR-CTC），通过强制同一输入语音经不同增强后得到的两个CTC分布保持一致来改进性能。深入分析表明：1）其在随机子模型对之间进行自蒸馏；2）通过在时间掩码区域进行掩码预测学习上下文表示，尤其在增加时间掩码量时效果更显著；3）抑制极端尖锐的CTC输出分布，降低过拟合，提升泛化能力。在LibriSpeech、Aishell-1和GigaSpeech数据集上的大量实验验证了该方法的有效性，显著提升了原始CTC性能，达到与转换器或结合CTC与注意力编码器解码器（CTC/AED）相当的先进水平。代码已开源于https://github.com/k2-fsa/icefall。
原文摘要 · Abstract (English)
Connectionist Temporal Classification (CTC) is a widely used method for automatic speech recognition (ASR), renowned for its simplicity and computational efficiency. However, it often falls short in recognition performance. In this work, we propose the Consistency-Regularized CTC (CR-CTC), which enforces consistency between two CTC distributions obtained from different augmented views of the input speech mel-spectrogram. We provide in-depth insights into its essential behaviors from three perspectives: 1) it conducts self-distillation between random pairs of sub-models that process different augmented views; 2) it learns contextual representation through masked prediction for positions within time-masked regions, especially when we increase the amount of time masking; 3) it suppresses the extremely peaky CTC distributions, thereby reducing overfitting and improving the generalization ability. Extensive experiments on LibriSpeech, Aishell-1, and GigaSpeech datasets demonstrate the effectiveness of our CR-CTC. It significantly improves the CTC performance, achieving state-of-the-art results comparable to those attained by transducer or systems combining CTC and attention-based encoder-decoder (CTC/AED). We release our code at https://github.com/k2-fsa/icefall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。