arXiv:2504.20334eess.AS2025-04被引 2

无需分类器自由引导,实现流匹配语音合成的高效推理

Towards Flow-Matching-based TTS without Classifier-Free Guidance

  • 重构训练目标以直接逼近引导优化路径
  • 推理速度提升9倍,质量与基线相当
  • 适合实时语音合成场景,兼容现有采样策略

流匹配在现代文语转换系统中展现出强大生成能力,广泛用于高质量语音合成。为保证合成质量,当前方法普遍依赖推理阶段的分类器自由引导(CFG),但其需两次前向传播,带来显著计算开销,限制了实时应用。本文探索在流匹配型TTS中移除CFG以提升推理效率,同时保持性能。通过重构流匹配训练目标,直接逼近CFG优化轨迹,避免推理时无条件模型评估和引导调优,使计算开销减半。该方法可无缝集成至现有优化采样策略。我们在LibriTTS数据集上以F5-TTS模型验证,结果表明,相较基线模型,推理速度提升9倍,且语音质量相当。代码与模型将开源,支持复现与后续研究。

原文摘要 · Abstract (English)

Flow matching has demonstrated strong generative capabilities and has become a core component in modern Text-to-Speech (TTS) systems. To ensure high-quality speech synthesis, Classifier-Free Guidance (CFG) is widely used during the inference of flow-matching-based TTS models. However, CFG incurs substantial computational cost as it requires two forward passes, which hinders its applicability in real-time scenarios. In this paper, we explore removing CFG from flow-matching-based TTS models to improve inference efficiency, while maintaining performance. Specifically, we reformulated the flow matching training target to directly approximate the CFG optimization trajectory. This training method eliminates the need for unconditional model evaluation and guided tuning during inference, effectively cutting the computational overhead in half. Furthermore, It can be seamlessly integrated with existing optimized sampling strategies. We validate our approach using the F5-TTS model on the LibriTTS dataset. Experimental results show that our method achieves a 9$\times$ inference speed-up compared to the baseline F5-TTS, while preserving comparable speech quality. We will release the code and models to support reproducibility and foster further research in this area.

语音合成流匹配高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。