自动选择剪枝策略,让模型更小更快且不丢精度。
Loss-Aware Automatic Selection of Structured Pruning Criteria for Deep Neural Network Acceleration
- 训练中同步剪枝,省去重训步骤,提升效率。
- 在CIFAR-10上,ResNet56/110剪掉52%计算量,准确率还更高。
- 无需手动调参,自动确定每层剪枝比例,适合部署优化。
结构化剪枝是压缩神经网络以适配资源受限边缘设备的成熟技术。本文提出一种损失感知的自动结构化剪枝准则选择方法(LAASP),实现深度神经网络的轻量化与加速。不同于传统三阶段流程(训练→剪枝→微调),该方法采用剪枝-训练一体化策略,合并剪枝与微调阶段,跳过初始训练环节。通过监测网络在小规模训练数据上的整体损失,自动从预设准则池中选择基于幅度或相似性的滤波器剪枝方式,并决定每轮剪枝的具体层。为缓解剪枝带来的精度骤降,每减少预设数量的浮点运算(FLOPs)后,网络会进行短时重训练。各层最优剪枝率自动确定,无需人工设定。在VGGNet和ResNet模型上对CIFAR-10与ImageNet数据集的实验表明,该方法有效:在CIFAR-10上,ResNet56与ResNet110的剪枝使计算量减少52%,同时显著提升准确率;在ImageNet上,ResNet50的计算量降低超42%,仅导致0.33%的top-5准确率下降。代码已开源。
原文摘要 · Abstract (English)
Structured pruning is a well-established technique for compressing neural networks, making it suitable for deployment in resource-limited edge devices. This paper presents an efficient Loss-Aware Automatic Selection of Structured Pruning Criteria (LAASP) for slimming and accelerating deep neural networks. The majority of pruning methodologies employ a sequential process consisting of three stages: 1) training, 2) pruning, and 3) fine-tuning, whereas the proposed pruning technique adopts a pruning-while-training approach that eliminates the first stage and integrates the second and third stages into a single cycle. The automatic selection of magnitude or similarity-based filter pruning criteria from a specified pool of criteria and the specific pruning layer at each pruning iteration is guided by the network's overall loss on a small subset of the training data. To mitigate the abrupt accuracy drop due to pruning, the network is retrained briefly after each reduction of a predefined number of floating-point operations (FLOPs). The optimal pruning rates for each layer in the network are automatically determined, eliminating the need for manual allocation of fixed or variable pruning rates for each layer. Experiments on the VGGNet and ResNet models on the CIFAR-10 and ImageNet benchmark datasets demonstrate the effectiveness of the proposed method. In particular, the ResNet56 and ResNet110 models on the CIFAR-10 dataset significantly improve the top-1 accuracy compared to state-of-the-art methods while reducing the network FLOPs by 52\%. Furthermore, the ResNet50 model on the ImageNet dataset reduces FLOPs by more than 42\% with a negligible 0.33\% drop in top-5 accuracy. The source code of this paper is publicly available online - https://github.com/ghimiredhikura/laasp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。