通过自适应重加权提升对抗迁移一致性，让小模型更稳定地学习大模型的抗攻击能力。
Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation
- 按样本级对抗可迁移性动态调整训练权重，无需额外计算开销。
- 在CIFAR-10/100和Tiny-ImageNet上，对AutoAttack的鲁棒性显著优于已有方法。
- 解决强教师不必然带来强学生的问题，适用于追求高效鲁棒模型的场景。
标准的最小-最大对抗训练框架中，对抗蒸馏旨在将大型稳健教师网络的对抗鲁棒性迁移到紧凑的学生网络。然而，现有工作常忽略最先进的稳健教师。通过广泛分析，我们发现更强的教师并不一定产生更稳健的学生——这一现象称为鲁棒饱和。尽管通常归因于容量差距，但我们表明此类解释不完整。相反，我们识别出对抗迁移性（即学生构造的对抗样本在教师上仍有效的比例）是成功鲁棒性迁移的关键因素。基于此洞察，我们提出样本级自适应对抗蒸馏（SAAD），通过测量的迁移性对训练样本进行重加权，且不增加额外计算成本。在CIFAR-10、CIFAR-100和Tiny-ImageNet上的实验表明，SAAD在对抗攻击鲁棒性方面持续优于先前方法。代码已公开于https://github.com/HongsinLee/saad。
原文摘要 · Abstract (English)
Adversarial distillation in the standard min-max adversarial training framework aims to transfer adversarial robustness from a large, robust teacher network to a compact student. However, existing work often neglects to incorporate state-of-the-art robust teachers. Through extensive analysis, we find that stronger teachers do not necessarily yield more robust students-a phenomenon known as robust saturation. While typically attributed to capacity gaps, we show that such explanations are incomplete. Instead, we identify adversarial transferability-the fraction of student-crafted adversarial examples that remain effective against the teacher-as a key factor in successful robustness transfer. Based on this insight, we propose Sample-wise Adaptive Adversarial Distillation (SAAD), which reweights training examples by their measured transferability without incurring additional computational cost. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that SAAD consistently improves AutoAttack robustness over prior methods. Our code is available at https://github.com/HongsinLee/saad.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。