提出HARP方法，在99%压缩率下仍保持模型准确性和抗攻击能力。
Holistic Adversarially Robust Pruning
- 按层动态学习最优剪枝策略，兼顾压缩率与鲁棒性。
- 99%模型尺寸缩减后，自然准确率和对抗鲁棒性均未下降。
- 适合对模型轻量化与安全性要求高的部署场景。
神经网络可通过移除冗余参数大幅缩小规模，但压缩常导致准确率下降且缺乏对抗鲁棒性。尽管近期已有进展，但仅在中等压缩率下有效。本文提出HARP方法，显著提升激进剪枝的性能。该方法从整体出发，为每层独立学习压缩策略：确定每层应剪枝的数量（压缩率）和具体参数（评分连接）。通过动态正则化进行微调，采用分步递增函数平衡目标——先侧重鲁棒性，再聚焦压缩率，最后均衡处理。所学策略使模型在原始尺寸减少99%的情况下，仍保持预训练模型的自然准确率和对抗鲁棒性。此外，发现跨层非均匀压缩具有关键影响。
原文摘要 · Abstract (English)
Neural networks can be drastically shrunk in size by removing redundant parameters. While crucial for the deployment on resource-constraint hardware, oftentimes, compression comes with a severe drop in accuracy and lack of adversarial robustness. Despite recent advances, counteracting both aspects has only succeeded for moderate compression rates so far. We propose a novel method, HARP, that copes with aggressive pruning significantly better than prior work. For this, we consider the network holistically. We learn a global compression strategy that optimizes how many parameters (compression rate) and which parameters (scoring connections) to prune specific to each layer individually. Our method fine-tunes an existing model with dynamic regularization, that follows a step-wise incremental function balancing the different objectives. It starts by favoring robustness before shifting focus on reaching the target compression rate and only then handles the objectives equally. The learned compression strategies allow us to maintain the pre-trained model natural accuracy and its adversarial robustness for a reduction by 99% of the network original size. Moreover, we observe a crucial influence of non-uniform compression across layers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。