用预训练生成式音频编码器提升语音降噪效率与音质。
Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoders
- 先提取噪声语音的嵌入，再用轻量编码器去噪，最后用声码器合成
- 在语音质量和说话人保真度上优于使用判别式编码器的模型
- 适合追求高效高质语音处理的研究者与工程师
近期研究探索了利用预训练模型生成的音频嵌入进行语音增强（SE），突破了传统时频掩码或信号预测方法。本文提出一种高效且可扩展的SE方法：首先使用预训练音频编码器从噪声语音中提取嵌入，再通过一个紧凑的编码器网络对嵌入进行去噪，最后由声码器从去噪嵌入合成干净语音。消融实验证明，结合预训练音频编码器和声码器的去噪编码器具有参数高效性。在语音增强与说话人保真度测试中，基于生成式音频编码器的系统性能优于采用判别式编码器的模型。主观听感测试进一步表明，本系统在感知质量上超越现有最先进模型。
原文摘要 · Abstract (English)
Recent research has delved into speech enhancement (SE) approaches that leverage audio embeddings from pre-trained models, diverging from time-frequency masking or signal prediction techniques. This paper introduces an efficient and extensible SE method. Our approach involves initially extracting audio embeddings from noisy speech using a pre-trained audioencoder, which are then denoised by a compact encoder network. Subsequently, a vocoder synthesizes the clean speech from denoised embeddings. An ablation study substantiates the parameter efficiency of the denoise encoder with a pre-trained audioencoder and vocoder. Experimental results on both speech enhancement and speaker fidelity demonstrate that our generative audioencoder-based SE system outperforms models utilizing discriminative audioencoders. Furthermore, subjective listening tests validate that our proposed system surpasses an existing state-of-the-art SE model in terms of perceptual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。