用CTC辅助大模型提升罕见词识别准确率
CTC-Assisted LLM-Based Contextual ASR
- 用CTC粗解码筛选相关关键词，融入大模型提示词
- 在LibriSpeech测试集上词错误率降至1.27%
- 支持两千个关键词偏置，适合实际场景应用
上下文语音识别或热词定制具有重要实用价值。尽管当前端到端（E2E）自动语音识别（ASR）系统表现优异，但在识别稀有词时仍存在困难。典型E2E上下文ASR模型通常架构复杂、解码机制繁琐，性能受限且易受干扰词影响。随着基于大语言模型（LLM）的ASR成为新主流，本文提出一种基于CTC辅助的LLM式上下文ASR模型，搭配高效过滤算法。通过使用粗粒度的CTC解码结果筛选潜在相关热词，并将其纳入LLM提示输入，模型在针对罕见长尾词识别的LibriSpeech test-clean和test-other数据集上分别取得1.27%/3.67%和2.72%/8.02%的词错误率（WER/B-WER），显著优于基线LLM-ASR模型，大幅超越其他相关工作。更令人瞩目的是，在大语言模型与所提过滤算法共同作用下，该模型在2000个偏置词条件下仍保持良好性能。
原文摘要 · Abstract (English)
Contextual ASR or hotword customization holds substantial practical value. Despite the impressive performance of current end-to-end (E2E) automatic speech recognition (ASR) systems, they often face challenges in accurately recognizing rare words. Typical E2E contextual ASR models commonly feature complex architectures and decoding mechanisms, limited in performance and susceptible to interference from distractor words. With large language model (LLM)-based ASR models emerging as the new mainstream, we propose a CTC-Assisted LLM-Based Contextual ASR model with an efficient filtering algorithm. By using coarse CTC decoding results to filter potential relevant hotwords and incorporating them into LLM prompt input, our model attains WER/B-WER of 1.27%/3.67% and 2.72%/8.02% on the Librispeech test-clean and test-other sets targeting on recognizing rare long-tail words, demonstrating significant improvements compared to the baseline LLM-based ASR model, and substantially surpassing other related work. More remarkably, with the help of the large language model and proposed filtering algorithm, our contextual ASR model still performs well with 2000 biasing words.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。