arXiv:2508.17265cs.CV2025-08被引 1

动态调整指导模型,提升学生模型对抗攻击的鲁棒性。

AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks

  • 设计自适应指导机制,让引导模型在训练中保持最优状态。
  • 在三个数据集上,显著提升目标模型对多种对抗攻击的防御能力。
  • 适合关注模型鲁棒性提升的开发者与研究者使用。

对抗蒸馏(AD)是一种知识蒸馏技术,可将教师深度神经网络(DNN)模型的鲁棒性迁移至轻量级目标(学生)模型,使学生模型性能优于独立训练的结果。部分先前工作采用小型可学习教师(引导)模型来增强学生模型的鲁棒性。由于可学习引导模型从零开始训练,难以在联合训练过程中维持其最佳状态以实现有效知识迁移。为此,本文提出一种新型自适应指导对抗训练方法(AdaGAT)。该方法通过动态调整引导模型的训练状态,向目标模型注入鲁棒性。具体地,我们设计了两个独立的损失函数,使引导模型更积极地参与反向传播,从而达到最优状态。我们在三个数据集(CIFAR-10、CIFAR-100、TinyImageNet)上进行了大量实验,以WideResNet-34-10作为目标模型。结果表明,在特定准确率范围内合理调整引导模型,可显著提升目标模型在多种对抗攻击下的鲁棒性,优于多种基线模型。

原文摘要 · Abstract (English)

Adversarial distillation (AD) is a knowledge distillation technique that facilitates the transfer of robustness from teacher deep neural network (DNN) models to lightweight target (student) DNN models, enabling the target models to perform better than only training the student model independently. Some previous works focus on using a small, learnable teacher (guide) model to improve the robustness of a student model. Since a learnable guide model starts learning from scratch, maintaining its optimal state for effective knowledge transfer during co-training is challenging. Therefore, we propose a novel Adaptive Guidance Adversarial Training (AdaGAT) method. Our method, AdaGAT, dynamically adjusts the training state of the guide model to install robustness to the target model. Specifically, we develop two separate loss functions as part of the AdaGAT method, allowing the guide model to participate more actively in backpropagation to achieve its optimal state. We evaluated our approach via extensive experiments on three datasets: CIFAR-10, CIFAR-100, and TinyImageNet, using the WideResNet-34-10 model as the target model. Our observations reveal that appropriately adjusting the guide model within a certain accuracy range enhances the target model's robustness across various adversarial attacks compared to a variety of baseline models.

对抗训练知识蒸馏鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。