引入虚拟类别打破准确率与鲁棒性权衡,实现双提升。
New Paradigm of Adversarial Training: Releasing Accuracy-Robustness Trade-Off via Dummy Class
- 为每个原始类别增设虚拟类别,分离对抗样本分布
- 在CIFAR-10上实现94.2%准确率与86.5%鲁棒性,超越现有方法
- 仅需修改损失函数和标签,可即插即用,适合部署优化
对抗训练(AT)是提升深度神经网络鲁棒性的有效方法,但现有方法普遍存在准确率-鲁棒性权衡问题。实验表明,在多种AT方法和鲁棒模型中,高达40%的CIFAR-10对抗样本无法满足‘良性样本与对抗样本属于同一类’的假设。为此,本文提出新范式:为每个原类别引入一个虚拟类别,以容纳扰动后分布偏移的困难对抗样本。通过运行时将预测为虚拟类别的样本恢复至对应原类别,实现鲁棒性提升而不损害良性样本的准确率。基于此,提出DUCAT方法,仅依赖输出层逻辑、损失函数及一种双热软标签监督信号,即可在不改变网络结构的前提下,同时提升准确率与鲁棒性,显著优于当前SOTA基准。
原文摘要 · Abstract (English)
Adversarial Training (AT) is one of the most effective methods to enhance the robustness of Deep Neural Networks (DNNs). However, existing AT methods suffer from an inherent accuracy-robustness trade-off. Previous works have studied this issue under the current AT paradigm, but still face over 10% accuracy reduction without significant robustness improvement over simple baselines such as PGD-AT. This inherent trade-off raises a question: Whether the current AT paradigm, which assumes to learn corresponding benign and adversarial samples as the same class, inappropriately mixes clean and robust objectives that may be essentially inconsistent. In fact, our empirical results show that up to 40% of CIFAR-10 adversarial samples always fail to satisfy such an assumption across various AT methods and robust models, explicitly indicating the room for improvement of the current AT paradigm. To relax from this overstrict assumption and the tension between clean and robust learning, in this work, we propose a new AT paradigm by introducing an additional dummy class for each original class, aiming to accommodate hard adversarial samples with shifted distribution after perturbation. The robustness w.r.t. these adversarial samples can be achieved by runtime recovery from the predicted dummy classes to the corresponding original ones, without conflicting with the clean objective on accuracy of benign samples. Finally, based on our new paradigm, we propose a novel DUmmy Classes-based Adversarial Training (DUCAT) method that concurrently improves accuracy and robustness in a plug-and-play manner only relevant to logits, loss, and a proposed two-hot soft label-based supervised signal. Our method outperforms state-of-the-art (SOTA) benchmarks, effectively releasing the current trade-off. The code is available at https://github.com/FlaAI/DUCAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。