arXiv:2409.01627cs.CV2024-09ECCV被引 9

通过动态修正教师错误,提升学生模型在对抗攻击下的准确率与鲁棒性。

Dynamic Guidance Adversarial Distillation with Enhanced Teacher Knowledge

  • 根据教师误判情况动态调整知识迁移重点
  • 在CIFAR10/100和Tiny ImageNet上提升学生模型准确率与抗攻击能力
  • 适合关注对抗训练中知识蒸馏优化的研究者

在对抗知识蒸馏(AD)中,从具备抗扰动能力的教师模型向较弱的学生模型进行策略性、精准的知识迁移至关重要。本文提出的动态引导对抗蒸馏(DGAD)框架直接应对样本重要性差异问题,聚焦于修正教师模型的误分类。DGAD采用误分类感知划分(MAP),动态调整蒸馏重点,通过引导至最可靠的教师预测来优化学习过程。此外,误差校正标签交换(ELS)对教师在干净输入与对抗扰动输入上的误分类进行纠正,提升知识传递质量。进一步地,预测一致性正则化(PCR)确保学生模型在干净与对抗输入下表现一致,显著增强整体鲁棒性。结合上述方法,DGAD在CIFAR10、CIFAR100和Tiny ImageNet数据集上,采用多种模型架构进行实验验证,显著提升了学生模型在干净数据上的准确率,并强化了其对复杂对抗威胁的防御能力,展现出作为提升学生模型鲁棒性与准确性的有力方案潜力。

原文摘要 · Abstract (English)

In the realm of Adversarial Distillation (AD), strategic and precise knowledge transfer from an adversarially robust teacher model to a less robust student model is paramount. Our Dynamic Guidance Adversarial Distillation (DGAD) framework directly tackles the challenge of differential sample importance, with a keen focus on rectifying the teacher model's misclassifications. DGAD employs Misclassification-Aware Partitioning (MAP) to dynamically tailor the distillation focus, optimizing the learning process by steering towards the most reliable teacher predictions. Additionally, our Error-corrective Label Swapping (ELS) corrects misclassifications of the teacher on both clean and adversarially perturbed inputs, refining the quality of knowledge transfer. Further, Predictive Consistency Regularization (PCR) guarantees consistent performance of the student model across both clean and adversarial inputs, significantly enhancing its overall robustness. By integrating these methodologies, DGAD significantly improves upon the accuracy of clean data and fortifies the model's defenses against sophisticated adversarial threats. Our experimental validation on CIFAR10, CIFAR100, and Tiny ImageNet datasets, employing various model architectures, demonstrates the efficacy of DGAD, establishing it as a promising approach for enhancing both the robustness and accuracy of student models in adversarial settings.

对抗蒸馏知识迁移鲁棒性提升模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。