用强化学习提升语音识别准确率，减少幻觉。
Group Relative Policy Optimization for Speech Recognition
- 基于规则奖励函数的分组相对策略优化方法
- 词错误率降低18.4%（相对），幻觉显著减少
- 适合需要高精度和鲁棒性的语音识别场景
语音识别正越来越多地采用大语言模型（LLMs）。这一趋势得益于LLMs良好的可扩展性、对大量标注与未标注语音及文本数据的利用能力、流式处理能力以及具备指令跟随特性的多任务学习能力。然而，传统基于下一词预测的目标在性能上存在局限，并易产生幻觉。本文提出将分组相对策略优化（GRPO）应用于自动语音识别（ASR）的强化学习框架中，设计简单规则化奖励函数引导策略更新。实验表明，该方法在词错误率上实现最高达18.4%的相对下降，显著减少幻觉，在域外数据集上更具鲁棒性，并有效支持领域迁移。
原文摘要 · Abstract (English)
Speech Recognition has seen a dramatic shift towards adopting Large Language Models (LLMs). This shift is partly driven by good scalability properties demonstrated by LLMs, ability to leverage large amounts of labelled, unlabelled speech and text data, streaming capabilities with auto-regressive framework and multi-tasking with instruction following characteristics of LLMs. However, simple next-token prediction objective, typically employed with LLMs, have certain limitations in performance and challenges with hallucinations. In this paper, we propose application of Group Relative Policy Optimization (GRPO) to enable reinforcement learning from human feedback for automatic speech recognition (ASR). We design simple rule based reward functions to guide the policy updates. We demonstrate significant improvements in word error rate (upto 18.4% relative), reduction in hallucinations, increased robustness on out-of-domain datasets and effectiveness in domain adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。