提出新方法实现扩散模型数据擦除，理论可靠且效果显著。
Data Unlearning in Diffusion Models
- 设计基于重要性采样的损失函数SISS，兼顾模型质量与数据删除
- 在CelebA-HQ和MNIST上达到质量与去记忆的帕累托最优
- 适用于版权保护场景，适合关注数据隐私的研究者
近期研究发现扩散模型会记忆并复现训练数据。随着大规模版权诉讼和GDPR等法规的推动，亟需从扩散模型中删除特定数据点。但重新训练成本过高，因此催生了数据去记忆（data unlearning）这一研究方向，旨在高效移除特定训练数据。现有概念去记忆方法依赖锚定提示/类别/分布，但在数据去记忆场景下不可用；通用机器去记忆技术则存在不稳定或无法有效去记忆的问题。为此，本文提出一类新型损失函数——减法重要性采样得分（Subtracted Importance Sampled Scores, SISS），首次在理论上提供保证，通过重要性采样构建加权组合损失，分别负责保持模型质量与删除目标数据。在CelebA-HQ和MNIST上，SISS实现了质量与去记忆强度的帕累托最优。在Stable Diffusion上，对近90%测试提示成功缓解了记忆现象。
原文摘要 · Abstract (English)
Recent work has shown that diffusion models memorize and reproduce training data examples. At the same time, large copyright lawsuits and legislation such as GDPR have highlighted the need for erasing datapoints from diffusion models. However, retraining from scratch is often too expensive. This motivates the setting of data unlearning, i.e., the study of efficient techniques for unlearning specific datapoints from the training set. Existing concept unlearning techniques require an anchor prompt/class/distribution to guide unlearning, which is not available in the data unlearning setting. General-purpose machine unlearning techniques were found to be either unstable or failed to unlearn data. We therefore propose a family of new loss functions called Subtracted Importance Sampled Scores (SISS) that utilize importance sampling and are the first method to unlearn data with theoretical guarantees. SISS is constructed as a weighted combination between simpler objectives that are responsible for preserving model quality and unlearning the targeted datapoints. When evaluated on CelebA-HQ and MNIST, SISS achieved Pareto optimality along the quality and unlearning strength dimensions. On Stable Diffusion, SISS successfully mitigated memorization on nearly 90% of the prompts we tested.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。