用对抗性混合样本来防止模型遗忘关键知识,提升数据删除安全性。
Adversarial Mixup Unlearning
- 设计生成器-遗忘器框架,通过对抗生成难样本增强遗忘训练。
- 在多个基准数据集上显著优于现有方法,避免灾难性遗忘。
- 适合关注隐私保护与模型可擦除性的研究人员使用。
机器遗忘是保护数据隐私的关键研究方向,旨在从模型中移除敏感信息。其中一大挑战是灾难性遗忘:删除特定数据时意外丢失重要知识,导致模型性能大幅偏离重新训练的结果。为此,我们提出一种新方法,利用合成的混合样本正则化遗忘过程,模拟易受灾难性影响的数据。核心为生成器-遗忘器框架 MixUnlearn:生成器对抗性地生成具有挑战性的混合样本,遗忘器基于这些合成数据有效遗忘目标信息。具体而言,引入新型对比损失训练生成器,使其生成能暴露应遗忘信息但不破坏关键知识的样本;遗忘器则在两个对比损失引导下联合处理合成与真实数据,确保精准遗忘且不丢失必要知识。大量实验表明,该方法在多个基准数据集上显著优于当前最优方案,为机器遗忘提供了稳健解决方案。本工作不仅深化了对遗忘机制的理解,也为基于混合增强的高效机器遗忘奠定了基础。
原文摘要 · Abstract (English)
Machine unlearning is a critical area of research aimed at safeguarding data privacy by enabling the removal of sensitive information from machine learning models. One unique challenge in this field is catastrophic unlearning, where erasing specific data from a well-trained model unintentionally removes essential knowledge, causing the model to deviate significantly from a retrained one. To address this, we introduce a novel approach that regularizes the unlearning process by utilizing synthesized mixup samples, which simulate the data susceptible to catastrophic effects. At the core of our approach is a generator-unlearner framework, MixUnlearn, where a generator adversarially produces challenging mixup examples, and the unlearner effectively forgets target information based on these synthesized data. Specifically, we first introduce a novel contrastive objective to train the generator in an adversarial direction: generating examples that prompt the unlearner to reveal information that should be forgotten, while losing essential knowledge. Then the unlearner, guided by two other contrastive loss terms, processes the synthesized and real data jointly to ensure accurate unlearning without losing critical knowledge, overcoming catastrophic effects. Extensive evaluations across benchmark datasets demonstrate that our method significantly outperforms state-of-the-art approaches, offering a robust solution to machine unlearning. This work not only deepens understanding of unlearning mechanisms but also lays the foundation for effective machine unlearning with mixup augmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。