用多模型知识蒸馏，让攻击更高效且更易迁移。
Teach Me to Trick: Exploring Adversarial Transferability via Knowledge Distillation
- 用多个不同教师模型蒸馏训练轻量学生模型。
- 生成攻击样本效率提升六倍，成功率接近集成方法。
- 低温训练+硬标签监督能显著提升攻击迁移性。
我们研究了从多个异构教师模型中进行知识蒸馏（KD），是否能增强可迁移对抗样本的生成能力。采用ResNet50和DenseNet-161作为教师，通过基于课程的切换与联合优化两种策略训练轻量级学生模型。随后利用FG、FGS和PGD攻击生成对抗样本，并在黑盒目标模型GoogLeNet上评估其效果。结果表明，多教师蒸馏的学生模型在攻击成功率上可媲美集成基线，同时将对抗样本生成时间缩短至原来的六分之一。消融实验进一步显示，较低温度设置及引入硬标签监督显著提升了攻击的迁移性。这些发现表明，知识蒸馏不仅是模型压缩手段，还可有效提升黑盒对抗攻击的效率与效果。
原文摘要 · Abstract (English)
We investigate whether knowledge distillation (KD) from multiple heterogeneous teacher models can enhance the generation of transferable adversarial examples. A lightweight student model is trained using two KD strategies: curriculum-based switching and joint optimization, with ResNet50 and DenseNet-161 as teachers. The trained student is then used to generate adversarial examples using FG, FGS, and PGD attacks, which are evaluated against a black-box target model (GoogLeNet). Our results show that student models distilled from multiple teachers achieve attack success rates comparable to ensemble-based baselines, while reducing adversarial example generation time by up to a factor of six. An ablation study further reveals that lower temperature settings and the inclusion of hard-label supervision significantly enhance transferability. These findings suggest that KD can serve not only as a model compression technique but also as a powerful tool for improving the efficiency and effectiveness of black-box adversarial attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。