用置信度指导大模型修正发音障碍者的语音识别错误。
Confidence-Guided Error Correction for Disordered Speech Recognition
- 将词级置信度嵌入大模型训练，引导其关注不确定区域。
- 在Speech Accessibility项目上相对降低10%误识率，TORGO上降47%。
- 适合需要提升残障人士语音识别准确率的研究与应用。
我们研究将大语言模型（LLMs）作为自动语音识别（ASR）的后处理模块，重点提升其对发音障碍语音的纠错能力。提出置信度引导提示方法，将词级不确定性估计直接嵌入LLM训练，增强跨说话人和数据集的鲁棒性与泛化能力。该方法能精准定位ASR中的不确定区域，减少过度纠正。我们微调了LLaMA 3.1模型，并与仅使用转录文本微调及事后置信度过滤的方法对比。实验显示，在Speech Accessibility Project的自然语音上，相比朴素的LLM纠错，本方法实现10%的相对词错率（WER）降低；在TORGO数据集上更达到47%的降低，验证了置信度感知微调在障碍语音场景下的有效性。
原文摘要 · Abstract (English)
We investigate the use of large language models (LLMs) as post-processing modules for automatic speech recognition (ASR), focusing on their ability to perform error correction for disordered speech. In particular, we propose confidence-informed prompting, where word-level uncertainty estimates are embedded directly into LLM training to improve robustness and generalization across speakers and datasets. This approach directs the model to uncertain ASR regions and reduces overcorrection. We fine-tune a LLaMA 3.1 model and compare our approach to both transcript-only fine-tuning and post hoc confidence-based filtering. Evaluations show that our method achieves a 10% relative WER reduction compared to naive LLM correction on the Speech Accessibility Project spontaneous speech and a 47% reduction on TORGO, demonstrating the effectiveness of confidence-aware fine-tuning for impaired speech.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。