通过强化学习提升语音大模型对生僻词的识别准确率
RLBR: Reinforcement Learning with Biasing Rewards for Contextual Speech Large Language Models
- 用偏置词奖励机制显式增强生僻词在训练中的权重
- 在100/500/1000个偏置词下，偏置词错误率降至0.59%/1.09%/1.36%
- 适合需要高精度识别专业术语或罕见词汇的应用场景
语音大语言模型在端到端语音理解与识别方面取得显著进展，但仍难以准确识别罕见词和领域特定术语。本文提出一种新型微调方法——带偏置奖励的强化学习（RLBR），通过专门设计的偏置词偏好奖励，在奖励计算中显式强调偏置词。同时引入参考文本感知机制，利用参考转录扩展强化学习的潜在轨迹探索空间。在不同偏置词列表规模下的LibriSpeech数据集实验表明，RLBR相较于强监督微调基线有显著性能提升，并持续优于近期多项方法。在测试集clean和other上，偏置词错误率（BWER）分别达到0.59%/2.11%、1.09%/3.24%和1.36%/4.04%（对应偏置词数为100、500、1000），且未牺牲整体词错误率（WER）。
原文摘要 · Abstract (English)
Speech large language models (LLMs) have driven significant progress in end-to-end speech understanding and recognition, yet they continue to struggle with accurately recognizing rare words and domain-specific terminology. This paper presents a novel fine-tuning method, Reinforcement Learning with Biasing Rewards (RLBR), which employs a specialized biasing words preferred reward to explicitly emphasize biasing words in the reward calculation. In addition, we introduce reference-aware mechanisms that extend the reinforcement learning algorithm with reference transcription to strengthen the potential trajectory exploration space. Experiments on the LibriSpeech corpus across various biasing list sizes demonstrate that RLBR delivers substantial performance improvements over a strong supervised fine-tuning (SFT) baseline and consistently outperforms several recently published methods. The proposed approach achieves excellent performance on the LibriSpeech test-clean and test-other sets, reaching Biasing Word Error Rates (BWERs) of 0.59% / 2.11%, 1.09% / 3.24%, and 1.36% / 4.04% for biasing list sizes of 100, 500, and 1000, respectively, without compromising the overall WERs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。