提出两种新正则化方法,提升深度神经网络对抗攻击的鲁棒性。
RegMix: Adversarial Mutual and Generalization Regularization for Enhancing DNN Robustness
- 用分解的KL散度损失实现对抗互学习,灵活控制优化过程
- 引入干净样本目标分布,显著增强模型泛化与鲁棒性
- 适用于提升对抗训练中模型的防御能力,尤其适合安全敏感场景
对抗训练是应对对抗攻击最有效的防御方法。现有方法多采用交叉熵损失和均方误差(MSE)作为正则化目标,但MSE在训练过程中强制两个输出分布过于均匀,限制了其在对抗训练中的鲁棒性。为此,我们重新审视互学习思想(最初用于知识蒸馏),提出两种专为对抗训练设计的新正则化策略:(i) 加权对抗互学习正则化,通过分解的对抗互KL散度损失,允许为主任务和辅助任务分配不等权重,灵活控制优化过程;(ii) 对抗泛化正则化,在对抗训练目标中引入额外的干净样本目标分布,提升模型泛化能力并增强鲁棒性。大量实验表明,所提方法相比现有基于正则化的对抗训练方法,在对抗鲁棒性上均有显著提升。
原文摘要 · Abstract (English)
Adversarial training is the most effective defense against adversarial attacks. The effectiveness of the adversarial attacks has been on the design of its loss function and regularization term. The most widely used loss function in adversarial training is cross-entropy and mean squared error (MSE) as its regularization objective. However, MSE enforces overly uniform optimization between two output distributions during training, which limits its robustness in adversarial training scenarios. To address this issue, we revisit the idea of mutual learning (originally designed for knowledge distillation) and propose two novel regularization strategies tailored for adversarial training: (i) weighted adversarial mutual regularization and (ii) adversarial generalization regularization. In the former, we formulate a decomposed adversarial mutual Kullback-Leibler divergence (KL-divergence) loss, which allows flexible control over the optimization process by assigning unequal weights to the main and auxiliary objectives. In the latter, we introduce an additional clean target distribution into the adversarial training objective, improving generalization and enhancing model robustness. Extensive experiments demonstrate that our proposed methods significantly improve adversarial robustness compared to existing regularization-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。