arXiv:2510.04058cs.LG2025-10被引 1

在数据受限时,让扩散模型遗忘不良内容生成能力。

Unlearning in Diffusion models under Data Constraints: A Variational Inference Approach

  • 基于变分推断框架,仅用部分不良数据微调模型。
  • 在不损失图像质量前提下,显著降低不良内容生成概率。
  • 适合需隐私保护或内容安全的生成模型部署场景。

为确保扩散模型在各领域负责任、安全地部署,需控制其生成内容,因其可能产生暴力、低俗输出。现有机器遗忘方法虽可从预训练生成模型中移除特定训练数据,但在全量训练数据不可访问的数据受限场景下效果不佳。本文提出一种新方法——变分扩散遗忘(VDU),可在仅访问含不良特征的子集数据条件下,有效防止预训练扩散模型生成不良内容。该方法基于变分推断框架,优化包含可塑性诱导项与稳定性正则项的损失函数:前者降低不良数据点的对数似然,后者通过参数空间正则化保障生成质量。我们在多任务上验证了方法有效性:在MNIST、CIFAR-10和tinyImageNet上对无条件去噪扩散概率模型(DDPM)进行类别遗忘;在基于LAION-5B训练的Stable Diffusion模型上实现高阶特征遗忘。

原文摘要 · Abstract (English)

For a responsible and safe deployment of diffusion models in various domains, regulating the generated outputs from these models is desirable because such models could generate undesired, violent, and obscene outputs. To tackle this problem, recent works use machine unlearning methodology to forget training data points containing these undesired features from pre-trained generative models. However, these methods proved to be ineffective in data-constrained settings where the whole training dataset is inaccessible. Thus, the principal objective of this work is to propose a machine unlearning methodology that can prevent the generation of outputs containing undesired features from a pre-trained diffusion model in such a data-constrained setting. Our proposed method, termed as Variational Diffusion Unlearning (VDU), is a computationally efficient method that only requires access to a subset of training data containing undesired features. Our approach is inspired by the variational inference framework with the objective of minimizing a loss function consisting of two terms: plasticity inducer and stability regularizer. Plasticity inducer reduces the log-likelihood of the undesired training data points, while the stability regularizer, essential for preventing loss of image generation quality, regularizes the model in parameter space. We validate the effectiveness of our method through comprehensive experiments for both class unlearning and feature unlearning. For class unlearning, we unlearn some user-identified classes from MNIST, CIFAR-10, and tinyImageNet datasets from a pre-trained unconditional denoising diffusion probabilistic model (DDPM). Similarly, for feature unlearning, we unlearn the generation of certain high-level features from a pre-trained Stable Diffusion model trained on LAION-5B dataset.

扩散模型机器遗忘内容安全数据约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。