动态调整网络宽度,让语音分离更高效。
Dynamic Slimmable Networks for Efficient Speech Separation
- 根据输入信号特点动态改变网络宽度
- 在WSJ0-2mix和WHAM!上提升性能效率比
- 适合资源受限设备上的实时语音分离
近年来语音分离的进展主要依赖于深度神经网络,但其高计算与内存需求限制了在资源受限设备上的部署。传统系统采用固定架构,对所有输入段保持相同计算量,即使面对简单片段(如静音或非重叠语音)也进行高强度处理,造成效率低下。为此,本文提出一种动态可裁剪网络(DSN),可根据输入信号特征自适应调整计算复杂度。DSN结合可裁剪网络(支持不同宽度运行)与轻量级门控模块,后者通过分析局部输入特征决定所需宽度。为平衡性能与效率,引入基于信号的复杂度损失,根据分段重建误差惩罚不必要的计算。在WSJ0-2mix和WHAM!数据集的干净与噪声双说话人混合语料上,DSN相较于独立训练的不同规模静态网络,实现了更优的性能-效率权衡。
原文摘要 · Abstract (English)
Recent progress in speech separation has been largely driven by advances in deep neural networks, yet their high computational and memory requirements hinder deployment on resource-constrained devices. A significant inefficiency in conventional systems arises from using static network architectures that maintain constant computational complexity across all input segments, regardless of their characteristics. This approach is sub-optimal for simpler segments that do not require intensive processing, such as silence or non-overlapping speech. To address this limitation, we propose a dynamic slimmable network (DSN) for speech separation that adaptively adjusts its computational complexity based on the input signal. The DSN combines a slimmable network, which can operate at different network widths, with a lightweight gating module that dynamically determines the required width by analyzing the local input characteristics. To balance performance and efficiency, we introduce a signal-dependent complexity loss that penalizes unnecessary computation based on segmental reconstruction error. Experiments on clean and noisy two-speaker mixtures from the WSJ0-2mix and WHAM! datasets show that the DSN achieves a better performance-efficiency trade-off than individually trained static networks of different sizes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。