提升语音转文字模型性能，通过一致性正则化增强鲁棒性
Transducer Consistency Regularization for Speech to Text Applications
- 对输入加噪声生成多视角数据，强制模型输出一致
- 在Librispeech上使词错误率降低4.3%（相对基准）
- 适合追求高精度语音识别的开发者和研究者
一致性正则化常用于促使模型对扰动输入产生一致表示，提升泛化能力，在多种基于交叉熵优化的语音任务中表现优异。然而，将其应用于广泛使用的基于变换器的语音模型仍具挑战，主要源于变换器优化准则中存在的巨大对齐空间，且并非所有对齐都同等促进模型优化。本文提出变换器一致性正则化（TCR），通过对输入施加频谱增强、丢弃等扰动生成不同数据视图，最小化输出分布差异。通过使用职业概率为变换器输出分布分配权重，仅让接近理想对齐的对齐结果参与学习。实验表明，该方法优于其他一致性正则化方案，在Librispeech数据集上相比强基线相对降低4.3%的词错误率（WER）。
原文摘要 · Abstract (English)
Consistency regularization is a commonly used practice to encourage the model to generate consistent representation from distorted input features and improve model generalization. It shows significant improvement on various speech applications that are optimized with cross entropy criterion. However, it is not straightforward to apply consistency regularization for the transducer-based approaches, which are widely adopted for speech applications due to the competitive performance and streaming characteristic. The main challenge is from the vast alignment space of the transducer optimization criterion and not all the alignments within the space contribute to the model optimization equally. In this study, we present Transducer Consistency Regularization (TCR), a consistency regularization method for transducer models. We apply distortions such as spec augmentation and dropout to create different data views and minimize the distribution difference. We utilize occupational probabilities to give different weights on transducer output distributions, thus only alignments close to oracle alignments would contribute to the model learning. Our experiments show the proposed method is superior to other consistency regularization implementations and could effectively reduce word error rate (WER) by 4.3\% relatively comparing with a strong baseline on the \textsc{Librispeech} dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。