用检索增强技术让语音识别适应新领域，无需训练数据。
Retrieval-Augmented Speech Recognition Approach for Domain Challenges
- 推理时检索领域文本，用大模型动态增强识别
- 在CSJ数据集上达到当前最佳准确率，无需全量训练数据
- 适合数据受限或保密的垂直领域语音应用
语音识别系统常因领域不匹配而表现不佳，尤其在真实场景中，受限于数据获取和保密要求，难以获得领域特定数据。受大语言模型检索增强生成（RAG）启发，本文提出一种基于大模型的检索增强语音识别方法，在推理阶段引入领域文本以提升识别性能。模型不依赖训练阶段的领域文本，而是学习如何利用提示中的文本信息来优化语音识别。借助RAG机制，该方法可高效访问本地可用的领域文档，实现便捷有效的领域适配。在CSJ数据库上的实验表明，该方法显著提升了语音识别准确率，并在未使用完整训练数据的情况下达到了当前最优性能。
原文摘要 · Abstract (English)
Speech recognition systems often face challenges due to domain mismatch, particularly in real-world applications where domain-specific data is unavailable because of data accessibility and confidentiality constraints. Inspired by Retrieval-Augmented Generation (RAG) techniques for large language models (LLMs), this paper introduces a LLM-based retrieval-augmented speech recognition method that incorporates domain-specific textual data at the inference stage to enhance recognition performance. Rather than relying on domain-specific textual data during the training phase, our model is trained to learn how to utilize textual information provided in prompts for LLM decoder to improve speech recognition performance. Benefiting from the advantages of the RAG retrieval mechanism, our approach efficiently accesses locally available domain-specific documents, ensuring a convenient and effective process for solving domain mismatch problems. Experiments conducted on the CSJ database demonstrate that the proposed method significantly improves speech recognition accuracy and achieves state-of-the-art results on the CSJ dataset, even without relying on the full training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。