让模型在不看数据的情况下,公平提升对各类别的抗攻击能力。
FERD: Fairness-Enhanced Data-Free Robustness Distillation
- 通过重加权和生成公平样本,均衡不同类别的鲁棒性。
- 在CIFAR-10上,最差类别鲁棒性提升15.1%(FGSM)和6.4%(AutoAttack)。
- 适合关注模型公平性与鲁棒性平衡的研究者与开发者。
数据自由鲁棒性蒸馏(DFRD)旨在不访问训练数据的情况下,将教师模型的鲁棒性迁移到学生模型。现有方法虽关注整体鲁棒性,却忽视了类别间鲁棒性的不公平问题,导致不同类别间鲁棒性差异显著。本文发现两个关键问题:(1) 学生模型在等比例类别数据下表现差异大;(2) 鲁棒性在不同攻击目标下不稳定。为此,提出首个公平增强型数据自由鲁棒性蒸馏(FERD)框架,通过调整对抗样本的比例与分布来解决。针对比例,采用鲁棒性引导的类别重加权策略,为鲁棒性较弱类别生成更多样本;针对分布,生成公平感知样本(FAEs),通过特征级预测均匀性约束抑制类别特异性非鲁棒特征;再基于FAEs构造统一目标对抗样本(UTAEs),施加均匀目标类别约束,避免攻击方向偏倚,使攻击目标均匀分布于所有类别,防止对特定脆弱类别的过拟合。在三个公开数据集上的实验表明,FERD在所有对抗攻击下均达到最优最差类别鲁棒性,例如在CIFAR-10上使用MobileNet-V2时,FGSM与AutoAttack下的最差类别鲁棒性分别提升15.1%和6.4%,在鲁棒性与公平性方面均表现卓越。
原文摘要 · Abstract (English)
Data-Free Robustness Distillation (DFRD) aims to transfer the robustness from the teacher to the student without accessing the training data. While existing methods focus on overall robustness, they overlook the robust fairness issues, leading to severe disparity of robustness across different categories. In this paper, we find two key problems: (1) student model distilled with equal class proportion data behaves significantly different across distinct categories; and (2) the robustness of student model is not stable across different attacks target. To bridge these gaps, we present the first Fairness-Enhanced data-free Robustness Distillation (FERD) framework to adjust the proportion and distribution of adversarial examples. For the proportion, FERD adopts a robustness-guided class reweighting strategy to synthesize more samples for the less robust categories, thereby improving robustness of them. For the distribution, FERD generates complementary data samples for advanced robustness distillation. It generates Fairness-Aware Examples (FAEs) by enforcing a uniformity constraint on feature-level predictions, which suppress the dominance of class-specific non-robust features, providing a more balanced representation across all categories. Then, FERD constructs Uniform-Target Adversarial Examples (UTAEs) from FAEs by applying a uniform target class constraint to avoid biased attack directions, which distribute the attack targets across all categories and prevents overfitting to specific vulnerable categories. Extensive experiments on three public datasets show that FERD achieves state-of-the-art worst-class robustness under all adversarial attack (e.g., the worst-class robustness under FGSM and AutoAttack are improved by 15.1\% and 6.4\% using MobileNet-V2 on CIFAR-10), demonstrating superior performance in both robustness and fairness aspects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。