通过音素增强融合提升语音识别纠错能力,尤其擅长区分同音词。
PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decoding
- 结合音素与上下文信息,增强对同音词的区分能力。
- 在五个数据集上降低偏倚词错误率,对同音词纠错效果更优。
- 适合需要快速推理且处理大量术语的场景,优于大模型方法。
端到端语音识别模型在识别生僻词时表现不佳。此前我们提出一种后处理方法ED-CEC,利用命名实体和技术术语等上下文信息提升识别准确率。尽管ED-CEC在纠正生僻词方面取得显著成效,但在处理发音相似但拼写不同的同音词时仍存在准确率低的问题。为此,本文在ED-CEC基础上提出音素增强的多模态融合方法PMF-CEC,以更好地区分目标生僻词与同音词。此外,发现先前的错误检测模块存在过度检测问题,因此引入保留概率机制,对置信度低于阈值的编辑操作进行过滤,保留原始结果以提高检测准确性。在五个数据集上的实验表明,PMF-CEC在保持合理推理速度的同时,进一步降低了偏倚词错误率,尤其在同音词纠正上优势明显。相比其他上下文引导方法及基于大语言模型的方法,本方法在推理速度更快、大规模偏置列表下更具鲁棒性。
原文摘要 · Abstract (English)
End-to-end automatic speech recognition (ASR) models often struggle to accurately recognize rare words. Previously, we introduced an ASR postprocessing method called error detection and context-aware error correction (ED-CEC), which leverages contextual information such as named entities and technical terms to improve the accuracy of ASR transcripts. Although ED-CEC achieves a notable success in correcting rare words, its accuracy remains low when dealing with rare words that have similar pronunciations but different spellings. To address this issue, we proposed a phoneme-augmented multimodal fusion method for context-aware error correction (PMF-CEC) method on the basis of ED-CEC, which allowed for better differentiation between target rare words and homophones. Additionally, we observed that the previous ASR error detection module suffers from overdetection. To mitigate this, we introduced a retention probability mechanism to filter out editing operations with confidence scores below a set threshold, preserving the original operation to improve error detection accuracy. Experiments conducted on five datasets demonstrated that our proposed PMF-CEC maintains reasonable inference speed while further reducing the biased word error rate compared with ED-CEC, showing a stronger advantage in correcting homophones. Moreover, our method outperforms other contextual biasing methods, and remains valuable compared with LLM-based methods in terms of faster inference and better robustness under large biasing lists.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。