一次训练循环完成高效结构化剪枝，精度高且省时。
One-Cycle Structured Pruning via Stability-Driven Subnetwork Search
- 用早期训练中的稳定性指标自动找最优子网络
- 在多个数据集上实现顶尖精度，训练成本更低
- 适合追求高效剪枝的模型部署开发者
现有结构化剪枝方法通常依赖多阶段训练，计算开销大。初始化剪枝虽能减轻负担，但性能常下降。为此，本文提出一种高效的单周期结构化剪枝框架，将预训练、剪枝与微调整合于单一训练周期内，不损失精度。核心思想是在训练初期通过基于范数的组重要性度量和结构稀疏正则化，识别最优子网络。提出一种新剪枝指标，通过连续训练周期间子网络相似性检测稳定剪枝时机。此外，组稀疏正则化加速收敛，进一步降低总训练时间。在CIFAR-10、CIFAR-100和ImageNet上，使用VGG、ResNet和MobileNet架构的大量实验表明，该方法在精度上达到当前最优水平，同时在训练成本上属于最高效的结构化剪枝框架之一。代码已开源：https://github.com/ghimiredhikura/OCSPruner。
原文摘要 · Abstract (English)
Existing structured pruning methods typically rely on multi-stage training procedures that incur high computational costs. Pruning at initialization aims to reduce this burden but often suffers from degraded performance. To address these limitations, we propose an efficient one-cycle structured pruning framework that integrates pre-training, pruning, and fine-tuning into a single training cycle without sacrificing accuracy. The key idea is to identify an optimal sub-network during the early stages of training, guided by norm-based group saliency criteria and structured sparsity regularization. We introduce a novel pruning indicator that detects a stable pruning epoch by measuring the similarity between pruning sub-networks across consecutive training epochs. In addition, group sparsity regularization accelerates convergence, further reducing overall training time. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet using VGG, ResNet, and MobileNet architectures demonstrate that the proposed method achieves state-of-the-art accuracy while being among the most efficient structured pruning frameworks in terms of training cost. Code is available at https://github.com/ghimiredhikura/OCSPruner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。