轻量级模型实现手机端全频段语音降噪,兼顾低延迟与高精度。
Lightweight DNN for Full-Band Speech Denoising on Mobile Devices: Exploiting Long and Short Temporal Patterns
- 基于改进UNet结构,结合短时与长时时间模式建模。
- 在48kHz全频带下实现实时因子低于0.02,性能超越现有方法。
- 专为移动端设计,适合资源受限场景的实时语音增强应用。
语音降噪(SD)是众多现代信号处理链中的关键任务,广泛应用于各类设备和日常生活中。尽管已有多种强大的深度神经网络(DNN)方法用于语音降噪,但针对资源受限平台(如移动设备)优化的方法较少。此外,多数现有方法未聚焦于全频带(FB)信号(采样率48 kHz)或低延迟场景。本文提出一种因果、低延迟、轻量化的全频带语音降噪DNN方法,同时利用短时与长时时间模式。该方法基于改进的UNet架构,引入回溯帧、卷积核的时间跨度扩展及循环神经网络,以捕捉信号与估计降噪掩码中的长短时依赖。DNN以因果帧间方式运行,输入为短时傅里叶变换(STFT)幅值,采用受MobileNet启发的倒置瓶颈结构,结合因果实例归一化进行通道归一化,部署于现代智能手机上时实时因子低于0.02。通过标准语音降噪指标与公开数据集评估,结果表明其在全频带低延迟场景下表现优于现有方法,达到更高的(SI-)SDR值。
原文摘要 · Abstract (English)
Speech denoising (SD) is an important task of many, if not all, modern signal processing chains used in devices and for everyday-life applications. While there are many published and powerful deep neural network (DNN)-based methods for SD, few are optimized for resource-constrained platforms such as mobile devices. Additionally, most DNN-based methods for SD are not focusing on full-band (FB) signals, i.e. having 48 kHz sampling rate, and/or low latency cases. In this paper we present a causal, low latency, and lightweight DNN-based method for full-band SD, leveraging both short and long temporal patterns. The method is based on a modified UNet architecture employing look-back frames, temporal spanning of convolutional kernels, and recurrent neural networks for exploiting short and long temporal patterns in the signal and estimated denoising mask. The DNN operates on a causal frame-by-frame basis taking as an input the STFT magnitude, utilizes inverted bottlenecks inspired by MobileNet, employs causal instance normalization for channel-wise normalization, and achieves a real-time factor below 0.02 when deployed on a modern mobile phone. The proposed method is evaluated using established speech denoising metrics and publicly available datasets, demonstrating its effectiveness in achieving an (SI-)SDR value that outperforms existing FB and low latency SD methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。