用可微分信号处理技术，让耳机实时变清晰
Improving Resource-Efficient Speech Enhancement via Neural Differentiable DSP Vocoder Refinement
- 用轻量网络预测音色、音高和周期性，驱动可微声码器生成语音
- 在不增加计算量前提下，语音可懂度提升4%，质量提升19%
- 适合资源受限的可穿戴设备，如智能眼镜
将语音增强（SE）系统部署于智能眼镜等可穿戴设备面临计算资源有限的挑战。尽管深度学习方法能实现高质量效果，但其高计算成本限制了在嵌入式平台的应用。本文提出一种高效的端到端语音增强框架，利用可微分数字信号处理（DDSP）声码器实现高质量语音合成。首先，一个紧凑的神经网络从噪声语音中预测增强后的声学特征：频谱包络、基频（F0）和周期性；这些特征输入至DDSP声码器以合成增强波形。系统通过短时傅里叶变换（STFT）与对抗损失进行端到端训练，实现特征与波形层面的直接优化。实验结果表明，该方法在不显著增加计算量的前提下，相较强基线模型在语音可懂度（STOI）上提升4%，在语音质量（DNSMOS）上提升19%，非常适合实时应用。
原文摘要 · Abstract (English)
Deploying speech enhancement (SE) systems in wearable devices, such as smart glasses, is challenging due to the limited computational resources on the device. Although deep learning methods have achieved high-quality results, their computational cost limits their feasibility on embedded platforms. This work presents an efficient end-to-end SE framework that leverages a Differentiable Digital Signal Processing (DDSP) vocoder for high-quality speech synthesis. First, a compact neural network predicts enhanced acoustic features from noisy speech: spectral envelope, fundamental frequency (F0), and periodicity. These features are fed into the DDSP vocoder to synthesize the enhanced waveform. The system is trained end-to-end with STFT and adversarial losses, enabling direct optimization at the feature and waveform levels. Experimental results show that our method improves intelligibility and quality by 4% (STOI) and 19% (DNSMOS) over strong baselines without significantly increasing computation, making it well-suited for real-time applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。