提出SORA方法,解决快速对抗训练中鲁棒性崩溃问题
SORA: Free Second-Order Attacks in Fast Adversarial Training

- 引入扰动可变性缓解灾难性过拟合
- 动态调整步长,实现高鲁棒性与高干净准确率
- 无需调参,通用性强,适合高效对抗训练
对抗训练(AT)是防御对抗样本的主流方法,但其高效单步变体常因灾难性过拟合(CO)导致对多步攻击的鲁棒性崩溃,尽管单步攻击性能优异。本文提出两个贡献:首先,形式化ε过拟合(EO),指出固定扰动幅度和方向会加剧CO,引入扰动可变性显著提升跨架构和数据集的鲁棒泛化能力;其次,提出PertAlign(扰动对齐)——一种理论基础扎实、计算开销极低的指标,通过测量攻击阶段间梯度对齐度预测CO发生。基于此,提出SORA,一种自适应步长的AT方法,根据损失曲面几何动态调整扰动。SORA能持续防止CO,实现顶尖的鲁棒性和干净准确率,并在单一固定超参数下跨数据集和架构泛化,这对快速对抗训练至关重要。在多种数据集和架构上的实验表明,SORA在鲁棒性上匹配或超越先前方法,同时具备更高干净准确率和更优效率。
原文摘要 · Abstract (English)
Adversarial Training (AT) is a leading defense against adversarial examples but often suffers from Catastrophic Overfitting (CO) in efficient single-step variants, where robustness to multi-step attacks collapses despite high single-step performance. We address this failure mode with two contributions. First, we formalize Epsilon Overfitting (EO), a perspective in which fixed perturbation magnitudes and directions exacerbate CO, and show that introducing perturbation variability significantly improves robust generalization across different architectures and datasets. Second, we propose PertAlign (Perturbation Alignment), a theoretically grounded, computationally negligible metric that predicts CO onset by measuring gradient alignment across attack stages. Leveraging these insights, we introduce SORA, an adaptive step-size AT method that dynamically adjusts perturbations based on loss surface geometry. SORA consistently prevents CO, achieves state-of-the-art robustness and clean accuracy, and generalizes across datasets and architectures using a single fixed set of hyperparameters, which is essential for applicability in fast AT. Extensive experiments on diverse datasets and architectures show that SORA matches or surpasses the robustness of prior methods while delivering higher clean accuracy and superior efficiency. Code is available at https://github.com/SecondOrderAT/SORA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。