arXiv:2502.04260cs.LG2025-02被引 1

提出新方法让生成模型彻底遗忘特定数据,同时保持其他性能。

Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retention

  • 用梯度上升解耦参数,使被遗忘样本成为分布外数据
  • 理论保证$(ε, δ)$-遗忘,实测在ImageNet和Places365上更优
  • 适合需要数据隐私保护的生成模型应用

机器遗忘允许参与者从训练好的模型中移除其数据以保护隐私与安全,但生成模型领域的研究仍较匮乏。现有图像到图像生成模型(I2I模型)的遗忘方法通常通过最小化高斯噪声与遗忘样本输出间的距离来实现,但我们认为该方法不合理——重训练模型能捕捉通用模式,不会生成等同于高斯噪声的输出。本文主张,遗忘后的模型应将遗忘样本视为分布外(OOD)数据,即不再识别或编码这些样本中的特定模式。为此,我们提出一种框架:通过梯度上升解耦模型参数,理论上确保遗忘样本在未学习模型中为分布外数据,并提供$(ε, δ)$-遗忘保证。随后在剩余样本上微调以维持性能,并引入攻击模型验证遗忘效果。在ImageNet-1K和Places365两个大规模数据集上的实验表明,本方法显著优于现有基线。此外,我们在CIFAR-10上对比了简单AutoEncoder与多种基线,证明本方法可达到接近重训练模型的性能。

原文摘要 · Abstract (English)

Machine Unlearning allows participants to remove their data from a trained machine learning model in order to preserve their privacy, and security. However, the machine unlearning literature for generative models is rather limited. The literature for image-to-image generative model (I2I model) considers minimizing the distance between Gaussian noise and the output of I2I model for forget samples as machine unlearning. However, we argue that the machine learning model performs fairly well on unseen data i.e., a retrained model will be able to catch generic patterns in the data and hence will not generate an output which is equivalent to Gaussian noise. In this paper, we consider that the model after unlearning should treat forget samples as out-of-distribution (OOD) data, i.e., the unlearned model should no longer recognize or encode the specific patterns found in the forget samples. To achieve this, we propose a framework which decouples the model parameters with gradient ascent, ensuring that forget samples are OOD for unlearned model with theoretical guarantee. We also provide $(ε, δ)$-unlearning guarantee for model updates with gradient ascent. The unlearned model is further fine-tuned on the remaining samples to maintain its performance. We also propose an attack model to ensure that the unlearned model has effectively removed the influence of forget samples. Extensive empirical evaluation on two large-scale datasets, ImageNet-1K and Places365 highlights the superiority of our approach. To show comparable performance with retrained model, we also show the comparison of a simple AutoEncoder on various baselines on CIFAR-10 dataset.

机器遗忘生成模型隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。