arXiv:2411.16901cs.CV2024-11ICML被引 3

用引力机制自动剪裁卷积神经网络,无需改结构也不用调参。

Deep Convolutional Neural Networks Structured Pruning via Gravity Regularization

  • 基于引力模型动态评估卷积核重要性,力大小与核质量及距离相关。
  • 剪掉受强引力影响的核,保留弱引力核,实现零权重删除。
  • 无需修改架构或长期微调,适合快速部署的模型压缩场景。

结构化剪裁是加速深度卷积神经网络(DCNN)的常用方法,但现有方法常需修改原始架构、实现复杂且需长时间微调。为此,我们提出一种受物理启发的新方法,将引力概念融入DCNN训练过程。引力大小与卷积核质量乘积成正比,与间距平方成反比。该力作用于卷积核:受弱引力者靠近非零权重,受强引力者远离并趋于零。强引力核权重被置零,可直接移除;弱引力核保留关键信息。本方法同时优化权重并排序重要性,避免复杂实现和长周期微调。我们在CIFAR数据集上对多个主流DCNN架构验证,结果优于或媲美现有方法。

原文摘要 · Abstract (English)

Structured pruning is a widely employed strategy for accelerating deep convolutional neural networks (DCNNs). However, existing methods often necessitate modifications to the original architectures, involve complex implementations, and require lengthy fine-tuning stages. To address these challenges, we propose a novel physics-inspired approach that integrates the concept of gravity into the training stage of DCNNs. In this approach, the gravity is directly proportional to the product of the masses of the convolution filter and the attracting filter, and inversely proportional to the square of the distance between them. We applied this force to the convolution filters, either drawing filters closer to the attracting filter (experiencing weaker gravity) toward non-zero weights or pulling filters farther away (subject to stronger gravity) toward zero weights. As a result, filters experiencing stronger gravity have their weights reduced to zero, enabling their removal, while filters under weaker gravity retain significant weights and preserve important information. Our method simultaneously optimizes the filter weights and ranks their importance, eliminating the need for complex implementations or extensive fine-tuning. We validated the proposed approach on popular DCNN architectures using the CIFAR dataset, achieving competitive results compared to existing methods.

模型剪枝卷积网络引力机制轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。