arXiv:2509.13667eess.AS2025-09中稿 · APSIPA ASC 2025被引 1

提出低延迟语音合成模型,兼顾音质与实时性。

A Distilled Low-Latency Neural Vocoder with Explicit Amplitude and Phase Prediction

  • 显式预测幅度和相位谱,通过iSTFT重建波形
  • 采用因果卷积降低延迟,效果接近非因果模型
  • 适合对响应速度要求高的语音应用

主流神经声码器多关注语音质量和生成速度,忽视了实时应用中至关重要的延迟问题。过高的延迟会导致用户交互明显延迟,严重影响体验,使系统难以用于实时场景。为此,本文提出DLL-APNet——一种经过知识蒸馏的低延迟神经声码器。该模型首先从输入的梅尔频谱显式预测幅度和相位谱,再通过逆短时傅里叶变换(iSTFT)重构语音波形。为降低延迟,采用因果卷积限制信息使用范围至当前及历史上下文。为缓解因果约束带来的音质下降,引入知识蒸馏策略:由预训练的非因果教师模型指导因果学生模型中间特征生成。实验表明,所提DLL-APNet在保持更低计算开销的同时,语音质量优于其他因果声码器,并达到主流非因果神经声码器水平,验证了其在高感知质量与低延迟之间的良好平衡。

原文摘要 · Abstract (English)

The majority of mainstream neural vocoders primarily focus on speech quality and generation speed, while overlooking latency, which is a critical factor in real-time applications. Excessive latency leads to noticeable delays in user interaction, severely degrading the user experience and rendering such systems impractical for real-time use. Therefore, this paper proposes DLL-APNet, a Distilled Low-Latency neural vocoder which first predicts the Amplitude and Phase spectra explicitly from input mel spectrogram and then reconstructs the speech waveform via inverse short-time Fourier transform (iSTFT). The DLL-APNet vocoder leverages causal convolutions to constrain the utilization of information to current and historical contexts, effectively minimizing latency. To mitigate speech quality degradation caused by causal constraints, a knowledge distillation strategy is proposed, where a pre-trained non-causal teacher vocoder guides intermediate feature generation of the causal student DLL-APNet vocoder. Experimental results demonstrate that the proposed DLL-APNet vocoder produces higher-quality speech than other causal vocoders, while requiring fewer computational resources. Furthermore, the proposed DLL-APNet vocoder achieves speech quality on par with mainstream non-causal neural vocoders, validating its ability to deliver both high perceptual quality and low latency.

语音合成低延迟知识蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。