用关键词感知损失提升稀有词语音识别效果
Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Function
- 设计关键词感知损失函数，强化稀有词训练
- 在NSC测试集上词错误率从29.71%降至11.81%
- 适合需要精准识别稀有词的语音系统优化
稀有词识别可通过将语音识别模型适配到包含这些词的合成数据来改善。进一步提升可通过上下文偏置实现：在模型架构中加入偏置模块并训练，以优先识别稀有词。尽管使用合成稀有词数据训练偏置模块比非稀有词数据更有效，但合成音频中的伪影可能导致过拟合。为此，本文改进基于TCPGen的上下文偏置方法，提出一种关键词感知损失函数，在训练偏置模块时额外关注偏置词。该损失包含用于偏置词预测的掩码交叉熵项，以及用于检测偏置词位置的二分类项，两者在推理时互补支持偏置词解码。通过将Whisper模型适配10小时合成数据，本方法在NSC Part 2测试集上的词错误率从29.71%降至11.81%。
原文摘要 · Abstract (English)
Rare word recognition can be improved by adapting ASR models to synthetic data that includes these words. Further improvements can be achieved through contextual biasing, which trains and adds a biasing module into the model architecture to prioritize rare words. While training the module on synthetic rare word data is more effective than using non-rare-word data, it can lead to overfitting due to artifacts in the synthetic audio. To address this, we enhance the TCPGen-based contextual biasing approach and propose a keyword-aware loss function that additionally focuses on biased words when training biasing modules. This loss includes a masked cross-entropy term for biased word prediction and a binary classification term for detecting biased word positions. These two terms complementarily support the decoding of biased words during inference. By adapting Whisper to 10 hours of synthetic data, our method reduced the word error rate on the NSC Part 2 test set from 29.71% to 11.81%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。