arXiv:2512.01054cs.LGcs.AI2025-12

动态调整遗忘强度,让生成模型更精准地删除特定内容。

Adaptive-lambda Subtracted Importance Sampled Scores in Machine Unlearning for DDPMs and VAEs

  • 让遗忘权重lambda随训练阶段和样本自动调整,提升灵活性。
  • 在扩增MNIST上,遗忘效果更强且保留数据生成质量更高。
  • 适合需要合规删除数据但又不想重训练的生成模型使用者。

机器遗忘对大型生成模型(如VAEs、DDPMs)至关重要,以满足“被遗忘权”要求并防止不希望的内容生成,而无需代价高昂的重新训练。现有方法(如扩散模型中的Static-lambda SISS)依赖固定混合权重lambda,因未考虑样本和训练阶段差异,效果不佳。本文提出Adaptive-lambda SISS,将lambda视为潜在变量,在每步训练中动态推断。通过轻量级推理网络,基于即时的SISS损失项(保留/遗忘损失及其梯度)的上下文特征,参数化lambda的后验分布。联合优化扩散模型与lambda推断机制,通过变分目标实现显著更优的权衡。进一步将自适应λ原则扩展至基于得分的遗忘,引入多类得分遗忘蒸馏(multi-class Score Forgetting Distillation)。此外提出两个新方向:(i) 结合无数据效率的得分遗忘蒸馏与直接梯度控制的SISS的混合目标;(ii) 将遗忘建模为序列决策过程,使用强化学习学习最优策略,状态空间定义为模型对遗忘集的记忆。在扩增MNIST基准上的实验表明,Adaptive-lambda SISS显著优于原始静态lambda方法,实现更强的遗忘类消除,同时更好保持保留集生成质量。

原文摘要 · Abstract (English)

Machine Unlearning is essential for large generative models (VAEs, DDPMs) to comply with the right to be forgotten and prevent undesired content generation without costly retraining. Existing approaches, such as Static-lambda SISS for diffusion models, rely on a fixed mixing weight lambda, which is suboptimal because the required unlearning strength varies across samples and training stages. We propose Adaptive-lambda SISS, a principled extension that turns lambda into a latent variable dynamically inferred at each training step. A lightweight inference network parameterizes an adaptive posterior over lambda, conditioned on contextual features derived from the instantaneous SISS loss terms (retain/forget losses and their gradients). This enables joint optimization of the diffusion model and the lambda-inference mechanism via a variational objective, yielding significantly better trade-offs. We further extend the adaptive-lambda principle to score-based unlearning and introduce a multi-class variant of Score Forgetting Distillation. In addition, we present two new directions: (i) a hybrid objective combining the data-free efficiency of Score Forgetting Distillation with the direct gradient control of SISS, and (ii) a Reinforcement Learning formulation that treats unlearning as a sequential decision process, learning an optimal policy over a state space defined by the model's current memory of the forget set. Experiments on an augmented MNIST benchmark show that Adaptive-lambda SISS substantially outperforms the original static-lambda SISS, achieving stronger removal of forgotten classes while better preserving generation quality on the retain set.

机器遗忘生成模型扩散模型去隐私

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。