提出高效分层剪枝法,显著降低CNN模型资源占用且不损失精度。
A Greedy Hierarchical Approach to Whole-Network Filter-Pruning in CNNs
- 分两层剪枝:先选要删的滤波器,再选最优剪枝层,全程用分类误差判断。
- 对ResNext101剪枝后内存从7.6GB降至1.5GB,FLOPS减少94%无精度损失。
- 适合部署在资源受限设备上的大型CNN模型轻量化,如移动端、嵌入式系统。
深度卷积神经网络在计算机视觉任务中表现优异,但其庞大的模型尺寸需要大量计算资源,因此对预训练的CNN进行冗余滤波器剪枝成为资源受限设备上构建高效模型的关键。全网络剪枝方法可对各层剪去不同比例的滤波器,灵活性更高。现有方法或因需计算每条滤波器剪枝后的损失而计算成本高,或依赖启发式/学习策略确定各层剪枝比例。本文提出一种两级分层剪枝方法,高效且以分类误差为最终判据。下层(滤波器剪枝)基于权重的线性近似,采用正交匹配追踪贪心选择和新型闭式误差准则的贪心反向剪枝,显著提升效率;上层(层选择)贪心选取最优剪枝层,采用两种全局标准:层间相对误差(HBGS)和最终分类误差(HBGTS)。在ResNet18、ResNet32、ResNet56、VGG16和ResNext101上均优于当前最佳方法。对ResNext101剪枝后内存由7.6 GB降至1.5 GB,FLOPS减少94%,在CIFAR-10上保持原精度。
原文摘要 · Abstract (English)
Deep convolutional neural networks (CNNs) have achieved impressive performance in many computer vision tasks. However, their large model sizes require heavy computational resources, making pruning redundant filters from existing pre-trained CNNs an essential task in developing efficient models for resource-constrained devices. Whole-network filter pruning algorithms prune varying fractions of filters from each layer, hence providing greater flexibility. Current whole-network pruning methods are either computationally expensive due to the need to calculate the loss for each pruned filter using a training dataset, or use various heuristic / learned criteria for determining the pruning fractions for each layer. This paper proposes a two-level hierarchical approach for whole-network filter pruning which is efficient and uses the classification loss as the final criterion. The lower-level algorithm (called filter-pruning) uses a sparse-approximation formulation based on linear approximation of filter weights. We explore two algorithms: orthogonal matching pursuit-based greedy selection and a greedy backward pruning approach. The backward pruning algorithm uses a novel closed-form error criterion for efficiently selecting the optimal filter at each stage, thus making the whole algorithm much faster. The higher-level algorithm (called layer-selection) greedily selects the best-pruned layer (pruning using the filter-selection algorithm) using a global pruning criterion. We propose algorithms for two different global-pruning criteria: (1) layer-wise relative error (HBGS), and (2) final classification error (HBGTS). Our suite of algorithms outperforms state-of-the-art pruning methods on ResNet18, ResNet32, ResNet56, VGG16, and ResNext101. Our method reduces the RAM requirement for ResNext101 from 7.6 GB to 1.5 GB and achieves a 94% reduction in FLOPS without losing accuracy on CIFAR-10.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。