arXiv:2606.12278cs.CVcs.LG2026-06

一周期内完成网络稀疏化,效果媲美多轮迭代方法。

Finding Sparse Subnetworks in One Training Cycle via Progressive Magnitude-Based Pruning

  • 训练中逐步增加稀疏度,按权重大小动态更新剪枝掩码。
  • 在ResNet-18上达72.9%稀疏度时准确率95.12%,超LTH的90.5%。
  • 适合追求高效剪枝、无需多轮训练的研究与工程场景。

神经网络剪枝通过移除不重要参数来减小模型规模,同时保持预测性能。尽管彩票票假设(LTH)表明,从合适初始化出发的稀疏子网络可达到密集网络的效果,但其迭代剪枝过程需多次完整训练周期。本文评估了渐进式幅度剪枝作为单周期替代方案:训练过程中按线性调度逐步提升稀疏度,并根据活跃权重幅度更新剪枝掩码。我们在CIFAR-10和MNIST数据集上对ResNet、VGG风格和LeNet架构进行系统实验,对比该方法与代表性迭代及初始化剪枝基线(包括LTH、SNIP、GraSP)。在CIFAR-10上,该方法在ResNet-18上实现72.9%稀疏度下的95.12%准确率,高于LTH报告的90.5%;在极端稀疏下,于VGG类架构达97%稀疏度时准确率93.13%(优于SNIP的约92.0%),在VGG-19上97.97%稀疏度达93.44%准确率(优于GraSP在98%稀疏度下的92.19%)。对ResNet-18的稀疏度-准确率分析显示,70–85%稀疏度区间内准确率仅比稠密基线低0.1个百分点。结果表明,渐进式幅度剪枝在评估条件下是一种高效的单周期神经网络稀疏化方法。

原文摘要 · Abstract (English)

Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance. Although the Lottery Ticket Hypothesis (LTH) shows that sparse subnetworks can match dense networks when trained from suitable initializations, its iterative pruning procedure requires multiple complete training cycles. This work evaluates progressive magnitude-based pruning as a single-cycle alternative. The method gradually increases sparsity during training using a linear schedule and updates pruning masks based on active weight magnitudes. We conduct systematic experiments on CIFAR-10 and MNIST across ResNet, VGG-style, and LeNet architectures, comparing the proposed method with representative iterative and initialization-based pruning baselines, including LTH, SNIP, and GraSP. On CIFAR-10, the method achieves 95.12\% accuracy on ResNet-18 at 72.9\% sparsity, compared with 90.5\% reported for LTH. At extreme sparsity, it achieves 93.13\% accuracy on a VGG-like architecture at 97\% sparsity, compared with approximately 92.0\% for SNIP, and 93.44\% accuracy on VGG-19 at 97.97\% sparsity, compared with 92.19\% for GraSP at 98\% sparsity. A sparsity-accuracy analysis on ResNet-18 further shows that accuracy remains within 0.1 percentage points of the dense baseline across 70--85\% sparsity. These results indicate that progressive magnitude-based pruning provides an effective single-cycle approach for neural network sparsification under the evaluated settings.

网络剪枝稀疏化单周期深度学习压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。