提出C-SAM框架,让模型在剪枝时同时保持小体积和抗干扰能力。
Towards Compact and Robust DNNs via Compression-aware Sharpness Minimization
- 在剪枝掩码上做扰动,让模型结构更平坦
- 剪枝后鲁棒性最高提升42%,精度几乎不变
- 适合移动端部署需要小模型又强鲁棒性的场景
Sharpness-Aware Minimization(SAM)虽能提升深度神经网络对输入变化的鲁棒性,但与设备端部署所需的紧凑性关系尚不明确。直接剪枝SAM训练模型会损害鲁棒性,因连续参数空间的平坦性无法转化为离散剪枝后的鲁棒性;而剪枝后再用SAM则受限于早期的非鲁棒性剪枝结构。为此,我们提出压缩感知的尖锐度最小化(C-SAM),将尖锐度感知学习从参数扰动转向掩码扰动。通过训练时显式扰动剪枝掩码,C-SAM使损失曲面在模型结构上更平坦,从而发现同时兼顾紧凑性与鲁棒性的剪枝模式。在CelebA-HQ、Flowers-102和CIFAR-10-C上对ResNet-18、GoogLeNet和MobileNet-V2的实验表明,C-SAM持续优于强基线,在认证鲁棒性上最高提升42%,且任务精度接近未剪枝模型。
原文摘要 · Abstract (English)
Sharpness-Aware Minimization (SAM) has recently emerged as an effective technique for improving DNN robustness to input variations. However, its interplay with the compactness requirements of on-device DNN deployments remains less explored. Simply pruning a SAM-trained model can undermine robustness, since flatness in the continuous parameter space does not necessarily translate to robustness under the discrete structural changes induced by pruning. Conversely, applying SAM after pruning may be fundamentally constrained by architectural limitations imposed by an early, robustness-agnostic pruning pattern. To address this gap, we propose Compression-aware ShArpness Minimization (C-SAM), a framework that shifts sharpness-aware learning from parameter perturbations to mask perturbations. By explicitly perturbing pruning masks during training, C-SAM promotes a flatter loss landscape with respect to model structure, enabling the discovery of pruning patterns that simultaneously optimize model compactness and robustness to input variations. Extensive experiments on CelebA-HQ, Flowers-102, and CIFAR-10-C across ResNet-18, GoogLeNet, and MobileNet-V2 show that C-SAM consistently achieves higher certified robustness than strong baselines, with improvements of up to 42%, while maintaining task accuracy comparable to the corresponding unpruned models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。