通过样本难易度筛选数据子集,提升模型对抗鲁棒性。
The Easy Path to Robustness: Coreset Selection using Sample Hardness
- 用平均输入梯度范数衡量样本难易度,易样本更少受攻击影响
- 在标准训练和TRADES训练下,对抗准确率提升最高达7%和5%
- 方法不依赖模型,适用于多种场景,适合关注数据筛选的研究者
从数据中心视角设计对抗鲁棒模型,需识别对学习鲁棒特征至关重要的样本。虽然核心集选择可实现高效数据子集训练,但现有算法针对干净准确率设计,难以保持鲁棒性。本文提出将样本的对抗脆弱性与其‘难易度’关联,通过训练过程中的平均输入梯度范数(AIGN)量化难易度。实验表明,易样本(低AIGN)对抗脆弱性较低,位于决策边界较远处。基于此,我们提出EasyCore算法,仅保留低AIGN样本用于训练。实证显示,使用EasyCore选取数据训练的模型,在标准训练和TRADES对抗训练下,相比现有核心集方法,对抗准确率分别提升最高达7%和5%。由于AIGN是模型无关的数据属性,EasyCore是一种高效且通用的数据中心鲁棒性增强方法。
原文摘要 · Abstract (English)
Designing adversarially robust models from a data-centric perspective requires understanding which input samples are most crucial for learning resilient features. While coreset selection provides a mechanism for efficient training on data subsets, current algorithms are designed for clean accuracy and fall short in preserving robustness. To address this, we propose a framework linking a sample's adversarial vulnerability to its \textit{hardness}, which we quantify using the average input gradient norm (AIGN) over training. We demonstrate that \textit{easy} samples (with low AIGN) are less vulnerable and occupy regions further from the decision boundary. Leveraging this insight, we present EasyCore, a coreset selection algorithm that retains only the samples with low AIGN for training. We empirically show that models trained on EasyCore-selected data achieve significantly higher adversarial accuracy than those trained with competing coreset methods under both standard and adversarial training. As AIGN is a model-agnostic dataset property, EasyCore is an efficient and widely applicable data-centric method for improving adversarial robustness. We show that EasyCore achieves up to 7\% and 5\% improvement in adversarial accuracy under standard training and TRADES adversarial training, respectively, compared to existing coreset methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。