用合成语音训练语音识别，强化学习比传统方法更有效。
When Synthetic Speech Is All You Have: Better Call GRPO
- 采用无评论器的强化学习方法GRPO，优化合成语音下的语音识别。
- 相比监督微调，词错误率降低40%（36.71%→22.09%），组合方法达45%。
- 适合缺乏真实语音数据、依赖合成语音的金融等隐私敏感领域。
基于大语言模型的语音识别系统在银行等受监管领域面临隐私瓶颈：真实语音采集成本高且法律受限，合成文本转语音（TTS）成为替代方案。然而，合成语音与真实录音存在声学差异，现有研究多局限于监督微调（SFT）。本文改用强化学习，证明组相对策略优化（GRPO）能从相同合成语音中挖掘更大潜力。仅使用合成语音进行GRPO适应，使词错误率（WER）相较SFT降低40%（36.71%→22.09%）；结合SFT再用GRPO，进一步降至45%。分析表明，性能提升源于行为优化而非表征变化：GRPO通过改善停止校准减少插入错误，并通过更好锚定注意力提升语音-文本对齐，而早期层表示保持不变。当合成语音为主要资源时，强化学习应优于监督微调。
原文摘要 · Abstract (English)
LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40\% relative to SFT (36.71\%$\to$22.09\%), and an SFT-then-GRPO combination pushes this further to 45\%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。