解决语言模型语音合成训练与推理不一致问题，提升长文本合成质量。
Bridging the gap between training and inference in LM-based TTS models
- 采用教师强制与自运行混合训练，引入自生成词元。
- 通过结束符预测机制动态控制生成过程，减少错误终止。
- 有效缓解训练-推理偏差，适合长序列语音合成研究者。
近期文本到语音（TTS）技术表明，基于语言模型（LM）的系统在性能上可媲美传统方法。然而，训练时使用真实词元作为前缀预测下一个词元，而推理时这些词元不可用，造成训练与推理之间的差距，常被忽视。本文提出一种提示引导的混合训练方案，以缓解主流LM-TTS系统中的暴露偏差。核心思路是结合教师强制与自运行训练，将自生成词元引入训练过程，使训练模式更接近推理状态。此外，在训练中引入结束符（EOS）预测机制，检测错误序列终止并自适应调控自运行过程。实验全面评估了暴露偏差对LM-TTS的影响，结果表明该方法能有效缩小训练-推理差距，显著提升长文本语音合成质量。
原文摘要 · Abstract (English)
Recent advancements in text-to-speech (TTS) have shown that language model (LM) based systems offer competitive performance compared to traditional approaches. However, in training, TTS models use ground-truth (GT) tokens as prefixes to predict the next token, while in inference these tokens are not available, a gap between training and inference that is often neglected. In this study, we propose a prompt-guided hybrid training scheme to mitigate exposure bias in popular LM-based TTS systems. Our core idea is to adopt a hybrid training paradigm that combines teacher forcing with free running, thereby introducing self-generated tokens into the training process. This makes the training mode more consistent with inference, reducing the training-inference gap. In addition, we incorporate an EOS prediction mechanism during training to detect incorrect sequence termination and adaptively control the free running process. Experimental results provide a comprehensive evaluation of the impact of exposure bias on LM-based TTS, and demonstrate that our method effectively narrows the training-inference gap, thereby improving the quality of synthesized long-form speech.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。