用合成数据和语音信息提升罕见词纠错效果
LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context
- 用合成数据增强罕见词训练，提升模型泛化能力
- 结合ASR候选词与语音上下文，减少误纠正
- 在英日语数据上均降低错误率，适合低频词场景
基于大语言模型的生成式纠错（GER）已成为提升自动语音识别（ASR）性能的有效后处理方法。然而，由于训练数据有限，其在罕见或领域特定词汇上的表现不佳。现有方法主要依赖文本信息，忽略语音线索，导致过度纠正。为此，我们提出一种针对罕见词的新型LLM-based GER方法，通过生成包含罕见词的合成数据进行模型微调，并整合ASR的N-best候选结果与语音上下文信息，以缓解过纠正问题。实验结果表明，该方法不仅显著提升罕见词纠错效果，还在英语和日语数据集上同时降低词错误率（WER）和字符错误率（CER）。
原文摘要 · Abstract (English)
Generative error correction (GER) with large language models (LLMs) has emerged as an effective post-processing approach to improve automatic speech recognition (ASR) performance. However, it often struggles with rare or domain-specific words due to limited training data. Furthermore, existing LLM-based GER approaches primarily rely on textual information, neglecting phonetic cues, which leads to over-correction. To address these issues, we propose a novel LLM-based GER approach that targets rare words and incorporates phonetic information. First, we generate synthetic data to contain rare words for fine-tuning the GER model. Second, we integrate ASR's N-best hypotheses along with phonetic context to mitigate over-correction. Experimental results show that our method not only improves the correction of rare words but also reduces the WER and CER across both English and Japanese datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。