提出S2AP方法,通过降低评分空间的尖锐度,提升对抗剪枝的稳定性与鲁棒性。
S2AP: Score-space Sharpness Minimization for Adversarial Pruning
- 在评分空间中引入尖锐度最小化机制,通过扰动权重重要性分数优化剪枝掩码
- 在多个数据集和稀疏度下显著降低评分空间尖锐度,提升剪枝后模型鲁棒性
- 无需修改原有架构,可作为插件集成到现有对抗剪枝流程中,适合模型压缩研究者
对抗剪枝方法已成为在保持对抗鲁棒性的同时压缩神经网络的有效工具。这类方法通常遵循三步流程:(i) 预训练一个鲁棒模型,(ii) 通过为权重分配重要性分数并选择二值掩码进行剪枝,(iii) 微调剪枝后的模型。现有方法在掩码选择阶段通常通过最小化鲁棒损失来确定重要性分数,但该评分空间优化容易陷入损失曲面的尖锐局部极小值,导致掩码选择不稳定,进而削弱剪枝后的鲁棒性。为此,本文提出一种新型即插即用的对抗剪枝方法——评分空间尖锐度感知剪枝(S2AP)。该方法在掩码搜索阶段引入评分空间尖锐度最小化,通过扰动重要性分数并最小化对应鲁棒损失,从而稳定掩码选择过程。大量实验表明,S2AP能有效降低评分空间的尖锐度,显著提升对抗剪枝的稳定性与最终模型的鲁棒性,在多种数据集、模型结构及稀疏度设置下均表现优异。
原文摘要 · Abstract (English)
Adversarial pruning methods have emerged as a powerful tool for compressing neural networks while preserving robustness against adversarial attacks. These methods typically follow a three-step pipeline: (i) pretrain a robust model, (ii) select a binary mask for weight pruning, and (iii) finetune the pruned model. To select the binary mask, these methods minimize a robust loss by assigning an importance score to each weight, and then keep the weights with the highest scores. However, this score-space optimization can lead to sharp local minima in the robust loss landscape and, in turn, to an unstable mask selection, reducing the robustness of adversarial pruning methods. To overcome this issue, we propose a novel plug-in method for adversarial pruning, termed Score-space Sharpness-aware Adversarial Pruning (S2AP). Through our method, we introduce the concept of score-space sharpness minimization, which operates during the mask search by perturbing importance scores and minimizing the corresponding robust loss. Extensive experiments across various datasets, models, and sparsity levels demonstrate that S2AP effectively minimizes sharpness in score space, stabilizing the mask selection, and ultimately improving the robustness of adversarial pruning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。