用检索增强技术修复语音识别中的罕见实体错误，效果显著。
Retrieval Augmented Correction of Named Entity Speech Recognition Errors
- 构建实体向量库，通过查询候选错误文本匹配正确实体
- 在稀有音乐实体测试集上降低33%-39%的词错误率
- 适合语音助手等需精准识别罕见名词的应用场景
近年来，端到端自动语音识别（ASR）系统表现出极高的准确性和性能，但在训练数据中出现频率较低的实体名称方面仍存在较高错误率。与此同时，大语言模型（LLMs）在多种自然语言处理任务中展现出强大能力。当有相关知识数据库可用时，检索增强生成（RAG）与LLM结合已取得优异成果。本文提出一种类似RAG的技术，用于纠正语音识别中的实体名称错误。该方法利用向量数据库索引一组相关实体，在运行时根据可能出错的ASR候选文本生成查询，通过查询获取的实体与原始假设一并输入经微调的LLM，以实现纠错。最终，最佳系统在聚焦稀有音乐实体的合成测试集上实现了33%-39%的相对词错误率降低，且未在公开的多领域语音助手测试集STOP上出现性能退化。
原文摘要 · Abstract (English)
In recent years, end-to-end automatic speech recognition (ASR) systems have proven themselves remarkably accurate and performant, but these systems still have a significant error rate for entity names which appear infrequently in their training data. In parallel to the rise of end-to-end ASR systems, large language models (LLMs) have proven to be a versatile tool for various natural language processing (NLP) tasks. In NLP tasks where a database of relevant knowledge is available, retrieval augmented generation (RAG) has achieved impressive results when used with LLMs. In this work, we propose a RAG-like technique for correcting speech recognition entity name errors. Our approach uses a vector database to index a set of relevant entities. At runtime, database queries are generated from possibly errorful textual ASR hypotheses, and the entities retrieved using these queries are fed, along with the ASR hypotheses, to an LLM which has been adapted to correct ASR errors. Overall, our best system achieves 33%-39% relative word error rate reductions on synthetic test sets focused on voice assistant queries of rare music entities without regressing on the STOP test set, a publicly available voice assistant test set covering many domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。