提出低延迟扩散模型，实现实时语音增强。
Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency
- 用滑动窗口逐步加噪，按时间推进去噪。
- 0.3~1秒延迟下性能优于传统扩散模型。
- 适合实时语音处理场景，如通话、直播。
扩散模型在语音增强中表现优异，但推理计算成本高，难以用于实时流数据处理。本文将滑动窗口扩散框架适配至语音增强任务，通过时间递进方式对语音信号逐步加噪，越接近当前时刻的帧所加噪声越多。该方法以缓冲区大小为基准引入可调延迟，实现性能与延迟的权衡。实验表明，本方法优于标准扩散模型，可在GPU上高效运行，输入输出延迟在0.3至1秒之间。这是首个实用的在线扩散语音增强方案。
原文摘要 · Abstract (English)
Diffusion models are a class of generative models that have been recently used for speech enhancement with remarkable success but are computationally expensive at inference time. Therefore, these models are impractical for processing streaming data in real-time. In this work, we adapt a sliding window diffusion framework to the speech enhancement task. Our approach progressively corrupts speech signals through time, assigning more noise to frames close to the present in a buffer. This approach outputs denoised frames with a delay proportional to the chosen buffer size, enabling a trade-off between performance and latency. Empirical results demonstrate that our method outperforms standard diffusion models and runs efficiently on a GPU, achieving an input-output latency in the order of 0.3 to 1 seconds. This marks the first practical diffusion-based solution for online speech enhancement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。