通过渐进式剪枝,高效压缩CNN模型且保持精度。
Model Compression using Progressive Channel Pruning
- 逐轮从多层中剪掉少量通道,结合尝试-选择-剪枝三步流程。
- 在ImageNet和CIFAR-10上优于现有方法,剪枝后精度损失更小。
- 适用于迁移学习,利用源域标签与目标域伪标签提升效果。
本文提出一种简单有效的通道剪枝框架Progressive Channel Pruning(PCP),用于加速卷积神经网络(CNN)。与现有方法仅单次逐层剪枝不同,本框架在每轮迭代中从多个选定层中逐步剪掉少量通道,包含尝试、选择、剪枝三步流程。在尝试阶段,使用任一现有剪枝方法对某层剪枝,并基于验证集样本估计精度下降;在选择阶段,采用贪心策略自动选出整体精度损失最小的层集合;在剪枝阶段,对选中的层执行剪枝。进一步将PCP扩展至深度迁移学习模型如领域对抗神经网络(DANN),通过结合源域标签与目标域伪标签,有效缓解剪枝过程中的数据分布差异。在ImageNet和CIFAR-10两个基准数据集上的全面实验表明,无论在监督学习还是迁移学习场景下,该框架均优于现有剪枝方法。
原文摘要 · Abstract (English)
In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。