轻量级语音增强模型，实时运行仅需37k参数
LiSenNet: Lightweight Sub-band and Dual-Path Modeling for Real-Time Speech Enhancement
- 分频段下采样与双路径递归结构捕捉时频特征
- 噪声检测模块自适应处理，降低计算开销
- 参数量仅37k，MAC数56M，适合边缘设备
语音增强（SE）旨在从含噪信号中恢复干净语音波形，以提升语音质量和可懂度。尽管基于学习的方法性能远超传统方法，但其高计算复杂度和大模型尺寸严重限制了在低延迟、低资源边缘设备上的部署。本文提出一种面向实时应用的轻量级语音增强网络（LiSenNet）。设计了分频段下采样与上采样模块，以及双路径循环模块，分别用于捕获频带感知特征和时频模式。引入噪声检测器识别含噪区域，实现自适应增强并节省计算成本。相比近期依赖高资源的基线模型，所提LiSenNet仅需37,000个参数（为当前最优模型的一半）和每秒5600万次乘加操作（MAC），即可达到具有竞争力的性能。
原文摘要 · Abstract (English)
Speech enhancement (SE) aims to extract the clean waveform from noise-contaminated measurements to improve the speech quality and intelligibility. Although learning-based methods can perform much better than traditional counterparts, the large computational complexity and model size heavily limit the deployment on latency-sensitive and low-resource edge devices. In this work, we propose a lightweight SE network (LiSenNet) for real-time applications. We design sub-band downsampling and upsampling blocks and a dual-path recurrent module to capture band-aware features and time-frequency patterns, respectively. A noise detector is developed to detect noisy regions in order to perform SE adaptively and save computational costs. Compared to recent higher-resource-dependent baseline models, the proposed LiSenNet can achieve a competitive performance with only 37k parameters (half of the state-of-the-art model) and 56M multiply-accumulate (MAC) operations per second.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。