用大模型提升语音识别纠错效果,支持多系统适配和零样本使用。
ASR Error Correction using Large Language Models
- 利用ASR的N-best列表提供更丰富上下文,提升纠错准确率。
- 提出约束解码机制,使输出更贴近原始语音转录,避免无关改动。
- 可跨ASR系统通用,支持零样本纠错,适合快速部署到新场景。
错误纠正(EC)模型在提升自动语音识别(ASR)转录质量方面至关重要,无需访问底层模型即可增强可读性并实现领域适应。本文研究基于大语言模型(LLM)的纠错方法,针对常见的一最佳(1-best)ASR输出,提出使用ASR的N-best列表作为输入以提供更丰富的上下文信息。标准EC模型生成过程无约束,可能导致不一致结果,为此引入基于N-best列表或ASR格网的约束解码策略。此外,多数EC模型需针对特定ASR系统训练,更换系统即需重新训练。本文探索了EC模型对不同ASR系统输出的泛化能力,并进一步拓展至使用ChatGPT等大模型实现零样本纠错。在三个标准数据集上的实验表明,所提方法在基于转换器和注意力编码器-解码器的ASR系统上均有效,且可作为有效的模型集成手段。
原文摘要 · Abstract (English)
Error correction (EC) models play a crucial role in refining Automatic Speech Recognition (ASR) transcriptions, enhancing the readability and quality of transcriptions. Without requiring access to the underlying code or model weights, EC can improve performance and provide domain adaptation for black-box ASR systems. This work investigates the use of large language models (LLMs) for error correction across diverse scenarios. 1-best ASR hypotheses are commonly used as the input to EC models. We propose building high-performance EC models using ASR N-best lists which should provide more contextual information for the correction process. Additionally, the generation process of a standard EC model is unrestricted in the sense that any output sequence can be generated. For some scenarios, such as unseen domains, this flexibility may impact performance. To address this, we introduce a constrained decoding approach based on the N-best list or an ASR lattice. Finally, most EC models are trained for a specific ASR system requiring retraining whenever the underlying ASR system is changed. This paper explores the ability of EC models to operate on the output of different ASR systems. This concept is further extended to zero-shot error correction using LLMs, such as ChatGPT. Experiments on three standard datasets demonstrate the efficacy of our proposed methods for both Transducer and attention-based encoder-decoder ASR systems. In addition, the proposed method can serve as an effective method for model ensembling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。