提出新方法同时提升模型准确率和抗干扰能力。
Robust Alignment: Harmonizing Clean Accuracy and Adversarial Robustness in Adversarial Training

- 通过固定边界样本的扰动强度,让模型更好学习有用模式。
- 引入语义对齐正则化,显著缓解准确率与鲁棒性矛盾。
- 在多个数据集上优于14种主流方法,实用性强。
对抗训练(AT)是构建鲁棒深度神经网络的有效方法,但存在干净准确率与对抗鲁棒性之间的权衡。本文首次发现:在决策边界附近的样本上改变输入扰动强度,对模型鲁棒性影响极小,这暴露了准确率与鲁棒性波动不一致的问题,根源在于输入空间与潜在空间的错位。为此,我们提出“鲁棒对齐”作为新的对抗训练目标,要求模型感知随输入扰动变化而保持标签不变。具体实现包括:对边界样本采用固定且较小的扰动强度,使扰动成为可学习的模式而非噪声;提出基于理论推导的领域插值一致性对抗正则化(DICAR),显式引入输入与潜在空间的语义对齐。结合二者,我们提出鲁棒对齐对抗训练(RAAT)方法,在CIFAR-10、CIFAR-100和Tiny-ImageNet上,使用ResNet-18、PreActResNet-18和WideResNet-28-10,均超越四种常见基线及14种最新SOTA方法,有效缓解了准确率与鲁棒性之间的权衡问题。
原文摘要 · Abstract (English)
Adversarial Training (AT) is one of the most effective methods for developing robust deep neural networks (DNNs). However, AT faces a trade-off problem between clean accuracy and adversarial robustness. In this work, we reveal a surprising phenomenon for the first time: Varying input perturbation intensities for training samples near decision boundaries in AT have minimal impact on model robustness. This finding directly exposes the inconsistency between accuracy and robustness score fluctuations, leading us to identify the misalignment between input and latent spaces as a critical driver of the robustness-accuracy trade-off. To mitigate this misalignment for harmonizing accuracy and robustness, we define Robust Alignment as a new AT target, encouraging the model perception to change with input perturbations provided the final label prediction remains unchanged, which can be achieved via two novel ideas. First, we suggest a reduced and fixed perturbation intensity for those boundary samples, which facilitates the model to utilize the perturbations as learnable patterns, instead of noises that complicate decision boundaries meaninglessly. Second, we propose a Domain Interpolation Consistency Adversarial Regularization (DICAR), based on rigorous theoretical derivations, which explicitly introduces semantic alignment between input and latent spaces into AT. Based on these two ideas, we end up with a new Robust Alignment Adversarial Training (RAAT) method, effectively harmonizing accuracy and robustness. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with ResNet-18, PreActResNet-18, and WideResNet-28-10 demonstrate the effectiveness of RAAT in improving the trade-off beyond four common baselines and a total of 14 related state-of-the-art (SOTA) works.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。