用大模型教小模型抗攻击，既强又准。
DARD: Dice Adversarial Robustness Distillation against Adversarial Attacks
- 大模型的抗攻击能力可蒸馏到小模型中
- 在相同架构下比对抗训练更鲁棒且准确
- 适合需要高效高安全性的部署场景
深度学习模型易受对抗样本攻击，在实际应用中带来重大安全挑战。尽管对抗训练（AT）是广泛采用的防御手段，但常导致自然数据上性能下降。近期研究发现，大模型相比小模型具有更强的鲁棒性。本文实证表明，这种鲁棒性可系统性地从大教师模型蒸馏至紧凑的学生模型。为此提出一种新方法DARD——Dice对抗鲁棒性蒸馏，通过定制知识蒸馏范式转移鲁棒性。同时提出DPGD，一种优化的对抗样本泛化方法以增强攻击效果。大量实验表明，DARD在相同架构下持续优于对抗训练网络，兼具更高鲁棒性和标准准确率。
原文摘要 · Abstract (English)
Deep learning models are vulnerable to adversarial examples, posing critical security challenges in real-world applications. While Adversarial Training (AT ) is a widely adopted defense mechanism to enhance robustness, it often incurs a trade-off by degrading performance on unperturbed, natural data. Recent efforts have highlighted that larger models exhibit enhanced robustness over their smaller counterparts. In this paper, we empirically demonstrate that such robustness can be systematically distilled from large teacher models into compact student models. To achieve better performance, we introduce Dice Adversarial Robustness Distillation (DARD), a novel method designed to transfer robustness through a tailored knowledge distillation paradigm. Additionally, we propose Dice Projected Gradient Descent (DPGD), an adversarial example generalization method optimized for effective attack. Our extensive experiments demonstrate that the DARD approach consistently outperforms adversarially trained networks with the same architecture, achieving superior robustness and standard accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。