攻击者通过污染蒸馏数据,让学生模型悄悄植入后门。
How to Backdoor the Knowledge Distillation
- 用含后门的对抗样本污染蒸馏数据,不改教师模型。
- 在多个数据集上实现高成功率、低被发现率的后门攻击。
- 首次揭示蒸馏过程的隐蔽漏洞,适合安全研究者关注。
知识蒸馏已成为现代机器学习系统的核心技术,通过将大型教师模型的知识迁移到更高效的学⽣模型中,实现模型压缩与加速。传统观点认为该过程安全,因后门攻击依赖被污染的训练数据和攻击者指定标签,而知识蒸馏仅使用干净教师模型的输出指导学生模型,从而天然抵御后门触发。本文挑战这一假设,提出一种新攻击方法:在蒸馏数据集中注入嵌入后门触发器的对抗样本,使学生模型在不改变教师模型的前提下被悄然攻陷。实验在多种数据集和攻击场景下验证了该方法的鲁棒性、隐蔽性和有效性。结果揭示了知识蒸馏中此前未被注意的安全漏洞,为未来防御此类攻击的研究提供了重要方向。
原文摘要 · Abstract (English)
Knowledge distillation has become a cornerstone in modern machine learning systems, celebrated for its ability to transfer knowledge from a large, complex teacher model to a more efficient student model. Traditionally, this process is regarded as secure, assuming the teacher model is clean. This belief stems from conventional backdoor attacks relying on poisoned training data with backdoor triggers and attacker-chosen labels, which are not involved in the distillation process. Instead, knowledge distillation uses the outputs of a clean teacher model to guide the student model, inherently preventing recognition or response to backdoor triggers as intended by an attacker. In this paper, we challenge this assumption by introducing a novel attack methodology that strategically poisons the distillation dataset with adversarial examples embedded with backdoor triggers. This technique allows for the stealthy compromise of the student model while maintaining the integrity of the teacher model. Our innovative approach represents the first successful exploitation of vulnerabilities within the knowledge distillation process using clean teacher models. Through extensive experiments conducted across various datasets and attack settings, we demonstrate the robustness, stealthiness, and effectiveness of our method. Our findings reveal previously unrecognized vulnerabilities and pave the way for future research aimed at securing knowledge distillation processes against backdoor attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。