arXiv:2410.07675cs.LGcs.AI2024-10被引 1

TRADES方法常高估模型抗攻击能力,研究揭示其原因并提出改进方案。

Adversarial Robustness Overestimation and Instability in TRADES

  • 通过分析梯度遮蔽现象,发现训练参数影响模型稳定性
  • 小批量、低beta值等设置易导致鲁棒性评估过高
  • 建议用输入加噪抑制过估计,适合关注模型真实鲁棒性的研究者

本文研究TRADES这一主流对抗训练方法中存在的概率鲁棒性过估计现象。实验发现,在多分类任务中,TRADES在PGD验证集上的准确率常显著高于AutoAttack测试准确率,表明对鲁棒性的过度估计,可能与梯度遮蔽有关。进一步分析显示,较小的批量大小、较低的beta值(控制鲁棒损失权重)、较高的学习率以及更高的类别复杂度(如CIFAR-100相比CIFAR-10)均增加过估计风险。通过分析一阶驻点条件(FOSC)、内层最大化和梯度信息,确认根本原因是梯度遮蔽。实验还发现某些不稳定的训练实例可恢复至无过估计状态。为此,建议调整参数设置或重新训练,并在FOSC得分超过阈值时向输入添加高斯噪声,从源头缓解TRADES及其他类似方法的鲁棒性过估计问题,提升评估可靠性。

原文摘要 · Abstract (English)

This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. We further analyze the parameters contributing to unstable models that lead to overestimation. Our findings indicate that smaller batch sizes, lower beta values (which control the weight of the robust loss term in TRADES), larger learning rates, and higher class complexity (e.g., CIFAR-100 versus CIFAR-10) are associated with an increased likelihood of robustness overestimation. By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it. Furthermore, our experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring our attempts at a solution. In addition to adjusting parameter settings to reduce instability or retraining when overestimation occurs, we recommend incorporating Gaussian noise in inputs when the FOSC score exceed the threshold. This method aims to mitigate robustness overestimation of TRADES and other similar methods at its source, ensuring more reliable representation of adversarial robustness during evaluation.

对抗训练鲁棒性评估梯度遮蔽模型稳定

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。