用ASR模型做奖励，让大语言模型语音合成更清晰自然。
Group Relative Policy Optimization for Text-to-Speech with Large Language Models
- 用现成的ASR模型计算奖励，无需额外训练奖励模型。
- 结合字符错误率和负对数似然，提升语音合成的可懂度和自然度。
- 适合想优化大模型语音合成效果的研究者和开发者。
本文提出一种基于GRPO的方法，通过使用现成的自动语音识别（ASR）模型为基于大语言模型（LLM）的文本到语音（TTS）模型提供奖励信号，以提升其性能。相比以往用于LLM-TTS的强化学习方法，本方法无需专门训练奖励模型。我们设计了一种复合奖励函数，融合了来自ASR模型的字符错误率（CER）与负对数似然（NLL），从而提供更丰富准确的奖励信号。将GRPO微调应用于预训练的LLM-TTS模型，并评估其零样本语音合成表现。实验结果表明，该方法显著提升了合成语音的可懂度与自然度。消融实验与进一步分析证实了两种奖励成分融合的有效性。
原文摘要 · Abstract (English)
This paper proposes a GRPO-based approach to enhance the performance of large language model (LLM)-based text-to-speech (TTS) models by deriving rewards from an off-the-shelf automatic speech recognition (ASR) model. Compared to previous reinforcement learning methods for LLM-based TTS, our method requires no dedicated model for reward computation or training. Moreover, we design a composite reward function that combines character error rate (CER) with negative log-likelihood (NLL) obtained from the ASR model, providing more informative and accurate reward signals. We apply GRPO fine-tuning to pre-trained LLM-based TTS models and evaluate their zero-shot TTS performance. Experimental results show that the proposed method substantially improves both the intelligibility and naturalness of synthesized speech. Ablation studies and further analyses confirm the effectiveness of integrating the two reward components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。