提出新训练方法，让模型对扰动的感知更平滑，缓解准确率与鲁棒性的矛盾。
Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
- 通过平滑输入扰动下的感知变化来优化决策边界
- 在多个数据集上优于12种先进方法，显著提升准确率与鲁棒性平衡
- 适合关注模型鲁棒性与泛化性能协同优化的研究者
对抗训练（AT）是提升深度神经网络鲁棒性的有效方法，但其常导致干净准确率与对抗鲁棒性之间的权衡。本文首次揭示一个反直觉现象：从感知一致性角度看，那些在对抗训练后仍能攻击鲁棒模型的困难对抗样本，反而比被成功防御的样本学得更好。因此，我们提出不同于以往观点的见解——正是对困难样本的过度学习破坏了决策边界，加剧了权衡问题。过度追求感知一致性会使模型将扰动视为噪声，忽略其中应被利用的信息，从而阻碍平滑的感知过渡。为此，我们提出一种名为鲁棒感知（Robust Perception）的新目标，并构建了鲁棒感知对抗训练（RPAT）方法，有效缓解了准确率-鲁棒性权衡。在CIFAR-10、CIFAR-100和Tiny-ImageNet上，使用ResNet-18、PreActResNet-18和WideResNet-34-10的实验表明，该方法超越四种常见基线及12种当前最先进方法。代码已开源。
原文摘要 · Abstract (English)
Adversarial Training (AT) is one of the most effective methods to train robust Deep Neural Networks (DNNs). However, AT creates an inherent trade-off between clean accuracy and adversarial robustness, which is commonly attributed to the more complicated decision boundary caused by the insufficient learning of hard adversarial samples. In this work, we reveal a counterintuitive fact for the first time: From the perspective of perception consistency, hard adversarial samples that can still attack the robust model after AT are already learned better than those successfully defended. Thus, different from previous views, we argue that it is rather the over-sufficient learning of hard adversarial samples that degrades the decision boundary and contributes to the trade-off problem. Specifically, the excessive pursuit of perception consistency would force the model to view the perturbations as noise and ignore the information within them, which should have been utilized to induce a smoother perception transition towards the decision boundary to support its establishment to an appropriate location. In response, we define a new AT objective named Robust Perception, encouraging the model perception to change smoothly with input perturbations, based on which we propose a novel Robust Perception Adversarial Training (RPAT) method, effectively mitigating the current accuracy-robustness trade-off. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with ResNet-18, PreActResNet-18, and WideResNet-34-10 demonstrate the effectiveness of our method beyond four common baselines and 12 state-of-the-art (SOTA) works. The code is available at https://github.com/FlaAI/RPAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。