arXiv:2502.04229cs.CRcs.AI2025-02被引 1

攻击者无需原始数据即可向蒸馏数据集植入后门,且不影响模型性能。

Dark Distillation: Backdooring Distilled Datasets without Accessing Raw Data

  • 通过重建类别概念原型,将后门注入蒸馏数据集
  • 攻击后模型在目标任务上保持高精度,后门触发率超90%
  • 适用于多种数据集和蒸馏方法,适合关注数据安全的研究者

数据蒸馏(DD)通过将大规模数据集压缩为小型合成数据集,提升训练效率并降低传输开销,使模型性能接近原始全量数据训练结果,已成为数据共享的主流方法。然而,其安全性尚未被充分研究。现有工作多假设攻击者在初始蒸馏阶段从原始数据注入后门,而本文首次揭示更现实的威胁:攻击者可拦截蒸馏数据分发过程,向蒸馏数据集植入后门并重新分发。我们证明,即便无原始数据访问权限,攻击仍可成功。具体地,本方法基于蒸馏数据集上训练的模型,重构每类的语义原型,并将其作为载体注入后门。同时设计混合损失函数,确保后门嵌入过程中不破坏原有优化轨迹,保留原始知识。大量实验表明,该攻击在多种原始数据集、蒸馏方法和下游训练策略下均有效,后门触发率超过90%,且生成恶意蒸馏数据耗时不足一分钟。

原文摘要 · Abstract (English)

Dataset distillation (DD) enhances training efficiency and reduces bandwidth by condensing large datasets into smaller synthetic ones. It enables models to achieve performance comparable to those trained on the raw full dataset and has become a widely adopted method for data sharing. However, security concerns in DD remain underexplored. Existing studies typically assume that malicious behavior originates from dataset owners during the initial distillation process, where backdoors are injected into raw datasets. In contrast, this work is the first to address a more realistic and concerning threat: attackers may intercept the dataset distribution process, inject backdoors into the distilled datasets, and redistribute them to users. While distilled datasets were previously considered resistant to backdoor attacks, we demonstrate that they remain vulnerable to such attacks. Furthermore, we show that attackers do not even require access to any raw data to inject the backdoors successfully. Specifically, our approach reconstructs conceptual archetypes for each class from the model trained on the distilled dataset. Backdoors are then injected into these archetypes to update the distilled dataset. Moreover, we ensure the updated dataset not only retains the backdoor but also preserves the original optimization trajectory, thus maintaining the knowledge of the raw dataset. To achieve this, a hybrid loss is designed to integrate backdoor information along the benign optimization trajectory, ensuring that previously learned information is not forgotten. Extensive experiments demonstrate that distilled datasets are highly vulnerable to backdoor attacks, with risks pervasive across various raw datasets, distillation methods, and downstream training strategies. Moreover, our attack method is efficient, capable of synthesizing a malicious distilled dataset in under one minute in certain cases.

数据安全后门攻击蒸馏模型鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。