提出首个面向语音分离的结构化剪枝方法，显著降低计算开销。
SepPrune: Structured Pruning for Efficient Deep Speech Separation
- 基于可微掩码实现通道级梯度驱动剪枝
- 剪枝后仅需1次微调即可恢复85%原始性能
- 相比从头训练快36倍，适合实时语音应用
尽管深度学习近年来大幅提升了语音分离性能，但多数研究仍侧重分离质量而忽视计算效率，而后者对实时低延迟语音处理至关重要。本文提出SepPrune，首个专为压缩深度语音分离模型、降低计算成本设计的结构化剪枝框架。该方法首先分析模型计算结构，识别计算负担最重的层；随后引入可微掩码策略，实现梯度驱动的通道选择；基于学习到的掩码，剪除冗余通道并微调剩余参数以恢复性能。大量实验表明，该可学习剪枝范式在语音分离模型中具有显著优势：经SepPrune剪枝的模型，仅用一次微调即可恢复预训练模型（训练数百轮）85%的性能，且收敛速度比从头训练快36倍。代码已开源：https://github.com/itsnotacie/SepPrune。
原文摘要 · Abstract (English)
Although deep learning has substantially advanced speech separation in recent years, most existing studies continue to prioritize separation quality while overlooking computational efficiency, an essential factor for low-latency speech processing in real-time applications. In this paper, we propose SepPrune, the first structured pruning framework specifically designed to compress deep speech separation models and reduce their computational cost. SepPrune begins by analyzing the computational structure of a given model to identify layers with the highest computational burden. It then introduces a differentiable masking strategy to enable gradient-driven channel selection. Based on the learned masks, SepPrune prunes redundant channels and fine-tunes the remaining parameters to recover performance. Extensive experiments demonstrate that this learnable pruning paradigm yields substantial advantages for channel pruning in speech separation models, outperforming existing methods. Notably, a model pruned with SepPrune can recover 85% of the performance of a pre-trained model (trained over hundreds of epochs) with only one epoch of fine-tuning, and achieves convergence 36$\times$ faster than training from scratch. Code is available at https://github.com/itsnotacie/SepPrune.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。