用关键词检索与强化学习，让大模型语音识别更准地听懂专有名词。
Contextual Biasing for LLM-Based ASR with Hotword Retrieval and Reinforcement Learning
- 先从海量词汇中筛选候选关键词，再注入大模型进行针对性训练。
- 在专用测试集上关键词错误率显著降低，通用语音识别准确率不受影响。
- 适合需要精准识别特定术语的场景，如医疗、金融语音转写。
基于大语言模型（LLM）的自动语音识别（ASR）在多种任务中表现优异，但在大规模词汇下对命名实体和关键词的上下文引导仍具挑战。本文提出一种可扩展的两阶段框架，融合关键词检索与LLM-ASR适配。首先，扩展全局-局部对比语言-音频预训练模型（GLCLAP），通过鲁棒性感知数据增强与模糊匹配，从大规模词汇库中检索出紧凑的前k个关键词候选集。其次，将检索到的候选词作为文本提示注入LLM-ASR模型，并采用生成式拒绝基策略优化（GRPO）进行微调，使用任务驱动奖励联合优化关键词识别与整体转录准确率。在聚焦关键词的测试集上实验表明，关键词错误率（KER）显著下降，同时在通用ASR基准上保持高句准确率，验证了该框架在大词汇上下文引导中的有效性。
原文摘要 · Abstract (English)
Large language model (LLM)-based automatic speech recognition (ASR) has recently achieved strong performance across diverse tasks, yet contextual biasing for named entities and hotwords under large vocabularies remains challenging. In this work, we propose a scalable two-stage framework that integrates hotword retrieval with LLM-ASR adaptation. First, we extend the Global-Local Contrastive Language-Audio pre-trained model (GLCLAP) to retrieve a compact top-k set of hotword candidates from a large vocabulary via robustness-aware data augmentation and fuzzy matching. Second, we inject the retrieved candidates as textual prompts into an LLM-ASR model and fine-tune it with Generative Rejection-Based Policy Optimization (GRPO), using a task-driven reward that jointly optimizes hotword recognition and overall transcription accuracy. Experiments on hotword-focused test sets show substantial keyword error rate (KER) reductions while maintaining sentence accuracy on general ASR benchmarks, demonstrating the effectiveness of the proposed framework for large-vocabulary contextual biasing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。