用可训练先验和固定点迭代，提升语音合成质量与速度。
Wave-Trainer-Fit: Neural Vocoder with Trainable Prior and Fixed-Point Iteration towards High-Quality Speech Generation from SSL features
- 引入可训练先验，从接近目标语音的噪声开始生成。
- 通过能量约束优化先验，减少生成步数，提升自然度。
- 对SSL特征提取深度不敏感，适合实际应用部署。
我们提出WaveTrainerFit，一种基于数据驱动特征（如自监督学习特征）生成高质量波形的神经声码器。该方法在WaveFit基础上改进：1. 引入可训练先验，使推理从接近目标语音的噪声开始，而非标准高斯噪声；2. 通过约束可训练先验匹配语音能量，实现参考感知增益调整。这些改进降低了从数据驱动特征建模波形的复杂度，使高质量波形生成仅需更少推断步骤。实验表明，WaveTrainerFit能生成更自然、说话人相似度更高的波形，且相比WaveFit所需迭代次数更少。此外，该方法在不同深度提取的SSL特征下均表现稳健。代码与预训练模型已公开于https://github.com/line/WaveTrainerFit。
原文摘要 · Abstract (English)
We propose WaveTrainerFit, a neural vocoder that performs high-quality waveform generation from data-driven features such as SSL features. WaveTrainerFit builds upon the WaveFit vocoder, which integrates diffusion model and generative adversarial network. Furthermore, the proposed method incorporates the following key improvements: 1. By introducing trainable priors, the inference process starts from noise close to the target speech instead of Gaussian noise. 2. Reference-aware gain adjustment is performed by imposing constraints on the trainable prior to matching the speech energy. These improvements are expected to reduce the complexity of waveform modeling from data-driven features, enabling high-quality waveform generation with fewer inference steps. Through experiments, we showed that WaveTrainerFit can generate highly natural waveforms with improved speaker similarity from data-driven features, while requiring fewer iterations than WaveFit. Moreover, we showed that the proposed method works robustly with respect to the depth at which SSL features are extracted. Code and pre-trained models are available from https://github.com/line/WaveTrainerFit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。