用超声波感知风速，提升单通道语音降噪效果
DeWinder: Single-Channel Wind Noise Reduction using Ultrasound Sensing
- 融合超声多普勒特征与语音信号进行多模态学习
- 在真实户外场景下使主流语音增强模型性能显著提升
- 适合户外语音采集、智能麦克风等需要抗风噪的场景
户外音频录制常受风噪声影响，由于其非平稳特性，单通道语音中的风噪声抑制仍是一大挑战。以往方法将风噪声视为一般背景噪声，未显式建模其特性。本文利用超声波作为辅助模态，显式感知气流并表征风噪声。提出一种多模态深度学习框架，融合超声多普勒特征与语音信号实现风噪声抑制。实验结果表明，DeWinder 能显著提升当前先进语音增强模型的降噪能力，在真实户外环境下表现优异。
原文摘要 · Abstract (English)
The quality of audio recordings in outdoor environments is often degraded by the presence of wind. Mitigating the impact of wind noise on the perceptual quality of single-channel speech remains a significant challenge due to its non-stationary characteristics. Prior work in noise suppression treats wind noise as a general background noise without explicit modeling of its characteristics. In this paper, we leverage ultrasound as an auxiliary modality to explicitly sense the airflow and characterize the wind noise. We propose a multi-modal deep-learning framework to fuse the ultrasonic Doppler features and speech signals for wind noise reduction. Our results show that DeWinder can significantly improve the noise reduction capabilities of state-of-the-art speech enhancement models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。