提出新方法提升数据蒸馏的抗攻击能力,解决精度与鲁棒性矛盾。
Mind Your Margin and Boundary: Are Your Distilled Datasets Truly Robust?

- 按鲁棒性边界重要性动态筛选对抗样本,聚焦低置信度攻击
- 通过对比学习增强类别间决策边界分离,提升抗扰动能力
- 在多个数据集上平均比现有方法鲁棒性高2.8%,适合安全敏感场景
数据蒸馏(DD)将大规模训练集压缩为小规模合成数据集以实现高效训练,但多数方法仅优化干净数据准确率,忽视鲁棒性。近期鲁棒性蒸馏方法虽有所改进,但仍存在精度-鲁棒性权衡问题:(i)对所有对抗扰动样本一视同仁,而实际鲁棒风险主要由接近零鲁棒边距的样本主导;(ii)未显式扩大攻击集中区域的类间决策边界分离。本文提出对抗感知课程-对比鲁棒蒸馏框架(C²R),从鲁棒边距视角推导扰动评分,近似每样本的鲁棒铰链损失,从而构建优先处理低边距对抗样本的课程机制。同时,引入类别平衡的对比鲁棒损失,在保持对抗不变性的同时显式拓宽跨类边界。在CIFAR-10/100、Tiny-ImageNet及多个ImageNet-1K子集上,面对六种攻击,C²R在平均鲁棒准确率上优于先前鲁棒蒸馏方法2.8%。
原文摘要 · Abstract (English)
Dataset distillation (DD) compresses a large training set into a small synthetic set for efficient training, but most DD methods optimize only clean accuracy and leave robustness uncontrolled. Recent robust DD methods improve robustness, yet they often suffer from a poor accuracy-robustness trade-off because they (i) treat all adversarially perturbed examples uniformly, despite robust risk being dominated by near-zero robust margins, and (ii) do not explicitly increase inter-class separation in the decision boundary where attacks concentrate. We present Contrastive Curriculum for Robust Dataset Distillation (C$^2$R), a framework that couples an attack-aware curriculum with a contrastive robustness objective. From a robust-margin perspective, we derive a perturbation score that approximates each sample's robust hinge, enabling a curriculum that prioritizes the smallest-margin adversaries that most directly drive robust error. In parallel, a class-balanced contrastive robustness loss enforces adversarial invariance while explicitly widening boundary separation across classes. Experiments on CIFAR-10/100, Tiny-ImageNet, and multiple ImageNet-1K subsets under six attacks show that C$^2$R achieves the best robust accuracy, outperforming prior robust DD by $2.8$% on average.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。