通过中间状态加速语音合成，提升自然度并缩短生成时间。
Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis
- 在粗到细框架中构建流路径的中间状态，替代纯噪声起始。
- 使用正交投影自适应定位状态位置，提升生成效率与质量。
- 轻量级模块适配多模型，支持自适应步长求解器显著提速。
我们提出浅层流匹配（SFM），一种新型机制，用于在粗到细生成范式下增强基于流匹配（FM）的文本到语音（TTS）模型。与传统FM模块不同，SFM不将弱生成器的粗粒度表示作为条件，而是从这些表示中构建流路径上的中间状态。训练时，采用正交投影方法自适应确定这些状态的时间位置，并基于单段分段流设计原则化构建策略。SFM推理从中间状态开始而非纯噪声，从而将计算聚焦于流路径的后半段。我们将SFM集成到多个TTS模型中，采用轻量级的SFM头。实验表明，SFM在客观与主观评估中均一致提升语音自然度，并在使用自适应步长常微分方程求解器时显著加速推理。演示与代码见 https://ydqmkkx.github.io/SFMDemo/。
原文摘要 · Abstract (English)
We propose Shallow Flow Matching (SFM), a novel mechanism that enhances flow matching (FM)-based text-to-speech (TTS) models within a coarse-to-fine generation paradigm. Unlike conventional FM modules, which use the coarse representations from the weak generator as conditions, SFM constructs intermediate states along the FM paths from these representations. During training, we introduce an orthogonal projection method to adaptively determine the temporal position of these states, and apply a principled construction strategy based on a single-segment piecewise flow. The SFM inference starts from the intermediate state rather than pure noise, thereby focusing computation on the latter stages of the FM paths. We integrate SFM into multiple TTS models with a lightweight SFM head. Experiments demonstrate that SFM yields consistent gains in speech naturalness across both objective and subjective evaluations, and significantly accelerates inference when using adaptive-step ODE solvers. Demo and codes are available at https://ydqmkkx.github.io/SFMDemo/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。