arXiv:2604.24332cs.LGcs.CR2026-04被引 1

提出动态引导策略,缓解快速对抗训练中的过拟合与准确率下降问题。

Mitigating Error Amplification in Fast Adversarial Training

论文配图:Mitigating Error Amplification in Fast Adversarial Training
图 1 · 摘自论文原文
  • 按样本置信度动态调整扰动强度和监督信号,避免错误放大。
  • 在CIFAR-10/100上实现比FAT高3.2%的鲁棒准确率,且干净输入性能更优。
  • 适合关注对抗鲁棒性与泛化能力平衡的研究者和工程应用。

快速对抗训练(FAT)通过促使网络学习对扰动不变的表示,有效提升模型鲁棒性。然而,FAT常出现灾难性过拟合(CO),即模型过度适应训练攻击而无法泛化到未见攻击。此外,以鲁棒性为导向的优化通常导致干净输入性能显著下降,且随着扰动预算增大,下降趋势加剧。本文通过分析引导强度对不同置信度样本的影响,发现低置信度样本是造成CO和鲁棒性-准确率权衡的主要原因。基于此,提出分布感知动态引导(DDG)策略:根据真实类别置信度动态缩放扰动幅度,引导样本趋向一致决策边界,减少虚假相关学习;同时根据样本预测状态动态调整监督信号,避免对错误信号过度强调。为缓解动态引导带来的梯度不稳定性,进一步设计加权正则约束。在标准基准测试上的大量实验表明,DDG有效缓解了灾难性过拟合和鲁棒性-准确率权衡问题。

原文摘要 · Abstract (English)

Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations. However, FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones. Moreover, robustness oriented optimization typically leads to notable performance degradation on clean inputs, and such degradation becomes increasingly severe as the perturbation budget grows. In this work, we conduct a comprehensive analysis of how guidance strength affects model performance by modulating perturbation and supervision levels across distinct confidence groups. The findings reveal that low confidence samples are the primary contributors to CO and the robustness accuracy trade off. Building on this insight, we propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal. Specifically, DDG scales the perturbation magnitude according to the sample confidence at the ground truth class, thereby guiding samples toward consistent decision boundaries while mitigating the influence of learning spurious correlations. Simultaneously, it dynamically adjusts the supervision signal based on the prediction state of each sample, preventing overemphasis on incorrect signals. To alleviate potential gradient instability arising from dynamic guidance, we further design a weighted regularization constraint. Extensive experiments on standard benchmarks demonstrate that DDG effectively alleviates both CO and the robustness accuracy trade off.

对抗训练鲁棒性动态引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。