提出基于通量与压力的可解释剪枝方法，实现精准稀疏控制。
Hyperflux: Pruning Reveals Importance
- 将剪枝建模为由通量和压力驱动的连续演化系统
- 在多个数据集上达到与主流方法相当的精度与稀疏度
- 提供微观到宏观的剪枝行为可解释性，适合模型优化研究者
网络剪枝用于降低大型神经网络的推理延迟和功耗。然而，多数方法仅关注实验结果，忽视对剪枝过程的理解。我们提出Hyperflux，一种新型$L_0$剪枝方法，将剪枝建模为由通量（权重移除的梯度响应）和压力（全局正则化，驱动权重向剪枝方向演化）决定的连续系统。通过该模型，Hyperflux在微观（权重再生/剪枝）和宏观（稀疏度收敛等）层面均具备可解释性。我们还设计了一种新颖的压力调度策略，可稳定实现目标稀疏度。Hyperflux在CIFAR-10、CIFAR-100和ImageNet数据集上，对ResNet-50、VGG-19和DeiT-T/S均取得了具有竞争力的结果。
原文摘要 · Abstract (English)
Network pruning is used to reduce inference latency and power consumption in large neural networks. However, most methods focus on empirical results at the expense of understanding the pruning process. We introduce Hyperflux, a novel $L_0$ method which models pruning as a continuously evolving system determined by flux, the gradient response to a weight's removal, and pressure, a global regularization driving weights toward pruning. By exploiting this model, Hyperflux's pruning behavior becomes understandable at both microscopic (weight regrowth/pruning) and macroscopic (sparsity convergence, etc.) levels. We also introduce a novel pressure scheduler that reliably targets desired sparsities. Hyperflux achieves competitive results with ResNet-50, VGG-19 and DeiT-T/S on CIFAR-10, CIFAR-100 and ImageNet datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。