对比单次与迭代剪枝，发现不同压缩率下各有优劣。
One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
- 通过系统对比单次与迭代剪枝，明确适用场景。
- 低剪枝率时单次更优，高剪枝率时迭代表现更好。
- 提出基于耐心度的混合剪枝法，适配实际部署需求。
剪枝是提升神经网络计算效率的核心压缩技术。传统上分为单次剪枝（一次训练+剪枝）和迭代剪枝（多轮剪枝以精细调整）。尽管迭代方法更受青睐，但其优势常被默认而非实证。本文首次系统性比较两类方法，涵盖结构化与非结构化剪枝、多种剪枝准则与模态。结果表明：低剪枝率下单次剪枝性能更优，高剪枝率则迭代剪枝表现更佳。基于此，我们提出基于耐心度的剪枝策略，并设计一种混合方法，在特定场景下超越传统方法。代码已开源：https://github.com/janumiko/pruning-benchmark。
原文摘要 · Abstract (English)
Pruning is a core technique for compressing neural networks to improve computational efficiency. This process is typically approached in two ways: one-shot pruning, which involves a single pass of training and pruning, and iterative pruning, where pruning is performed over multiple cycles for potentially finer network refinement. Although iterative pruning has historically seen broader adoption, this preference is often assumed rather than rigorously tested. Our study presents one of the first systematic and comprehensive comparisons of these methods, providing rigorous definitions, benchmarking both across structured and unstructured settings, and applying different pruning criteria and modalities. We find that each method has specific advantages: one-shot pruning proves more effective at lower pruning ratios, while iterative pruning performs better at higher ratios. Building on these findings, we advocate for patience-based pruning and introduce a hybrid approach that can outperform traditional methods in certain scenarios, providing valuable insights for practitioners selecting a pruning strategy tailored to their goals and constraints. Source code is available at https://github.com/janumiko/pruning-benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。