提出低延迟语音合成模型,兼顾音质与实时性。
A Distilled Low-Latency Neural Vocoder with Explicit Amplitude and Phase Prediction
- 显式预测幅度和相位谱,通过iSTFT重建波形
- 采用因果卷积降低延迟,效果接近非因果模型
- 适合对响应速度要求高的语音应用
主流神经声码器多关注语音质量和生成速度,忽视了实时应用中至关重要的延迟问题。过高的延迟会导致用户交互明显延迟,严重影响体验,使系统难以用于实时场景。为此,本文提出DLL-APNet——一种经过知识蒸馏的低延迟神经声码器。该模型首先从输入的梅尔频谱显式预测幅度和相位谱,再通过逆短时傅里叶变换(iSTFT)重构语音波形。为降低延迟,采用因果卷积限制信息使用范围至当前及历史上下文。为缓解因果约束带来的音质下降,引入知识蒸馏策略:由预训练的非因果教师模型指导因果学生模型中间特征生成。实验表明,所提DLL-APNet在保持更低计算开销的同时,语音质量优于其他因果声码器,并达到主流非因果神经声码器水平,验证了其在高感知质量与低延迟之间的良好平衡。
原文摘要 · Abstract (English)
The majority of mainstream neural vocoders primarily focus on speech quality and generation speed, while overlooking latency, which is a critical factor in real-time applications. Excessive latency leads to noticeable delays in user interaction, severely degrading the user experience and rendering such systems impractical for real-time use. Therefore, this paper proposes DLL-APNet, a Distilled Low-Latency neural vocoder which first predicts the Amplitude and Phase spectra explicitly from input mel spectrogram and then reconstructs the speech waveform via inverse short-time Fourier transform (iSTFT). The DLL-APNet vocoder leverages causal convolutions to constrain the utilization of information to current and historical contexts, effectively minimizing latency. To mitigate speech quality degradation caused by causal constraints, a knowledge distillation strategy is proposed, where a pre-trained non-causal teacher vocoder guides intermediate feature generation of the causal student DLL-APNet vocoder. Experimental results demonstrate that the proposed DLL-APNet vocoder produces higher-quality speech than other causal vocoders, while requiring fewer computational resources. Furthermore, the proposed DLL-APNet vocoder achieves speech quality on par with mainstream non-causal neural vocoders, validating its ability to deliver both high perceptual quality and low latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。