arXiv:2502.09125cs.CVcs.AI2025-02被引 2

基于类别信息的结构化稀疏剪枝,让模型更小更快还更准。

Enhanced Structured Lasso Pruning with Class-wise Information

  • 用信息瓶颈理论指导剪枝,保留类别相关统计信息。
  • 在CIFAR-10上剪掉85%参数,精度仍达94.10%,比原模型高0.14%。
  • 适合追求高精度轻量模型的开发者,尤其图像分类任务。

现代应用需要轻量化神经网络模型。现有剪枝方法多关注移除不重要滤波器,但忽略类别信息可能导致统计信息丢失。本文从信息瓶颈理论出发,利用精确的类别相关性设计结构化Lasso剪枝,提出两种并行新方案:带信息瓶颈的稀疏图结构化Lasso剪枝(sGLP-IB)与带信息瓶颈的稀疏树引导Lasso剪枝(sTLP-IB)。通过更精准的类别相关性评估进行滤波器剪枝。在三个数据集和六种模型结构上,相比多个先进方法表现最优。例如,在CIFAR-10上的VGG16模型,参数减少85%,计算量降低61%,准确率保持94.10%(较原模型提升0.14%);在ImageNet上使用ResNet架构,参数减少55%,准确率仅下降0.03%至76.12%。结果表明,该方法可在显著降低模型规模和计算开销的同时,维持甚至提升模型性能。

原文摘要 · Abstract (English)

Modern applications require lightweight neural network models. Most existing neural network pruning methods focus on removing unimportant filters; however, these may result in the loss of statistical information after pruning due to failing to consider the class-wise information. In this paper, we employ the structured lasso from the perspective of utilizing precise class-wise information for model pruning with the help of Information Bottleneck theory, which guides us to ensure the retention of statistical information before and after pruning. With these techniques, we propose two novel adaptive network pruning schemes in parallel: sparse graph-structured lasso pruning with Information Bottleneck (sGLP-IB) and sparse tree-guided lasso pruning with Information Bottleneck (sTLP-IB). The key component is that we prune the model filters utilizing sGLP-IB and sTLP-IB with more precise structured class-wise relatedness. Compared to multiple state-of-the-art methods, our approaches achieve the best performance across three datasets and six model structures on extensive experiments. For example, with the VGG16 model based on the CIFAR-10 dataset, we can reduce the parameters by 85%, decrease the FLOPs by 61%, and maintain an accuracy of 94.10% (0.14% better than the original). For large-scale ImageNet, we can reduce the parameters by 55% while keeping the accuracy at 76.12% (only drop 0.03%) using the ResNet architecture. In summary, we succeed in reducing the model size and computational resource usage while maintaining the effectiveness of accuracy.

模型剪枝轻量化信息瓶颈图像分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。