让扩散模型精准删除多个概念,还能保持生成质量。
Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization
- 用动态掩码和概念感知损失,分步清除多个知识
- 多概念遗忘后图像质量与语义一致性显著提升
- 适合需要移除版权内容或偏见的AI应用
文本到图像扩散模型在生成高质量图像方面取得显著进展,但其存储大量知识的能力在需选择性遗忘的场景下引发担忧,如移除版权内容、减少偏见或消除有害概念。现有去学习方法虽可去除特定概念,但在多概念遗忘时面临不稳定性、残留知识及生成质量下降等问题。为此,我们提出一种新框架——动态掩码结合概念感知损失,专为扩散模型中的多概念遗忘设计。动态掩码根据优化状态自适应更新梯度掩码,实现对相关权重的精准修改,避免干扰无关知识;概念感知损失通过类别层级对齐显式引导去学习过程,确保语义一致性;基于知识蒸馏的正则化损失则保障在连续去学习过程中已遗忘的概念不再恢复。我们在多种场景下开展实验,结果表明,该方法在遗忘效果、输出保真度和语义连贯性上均优于现有技术,尤其在多概念场景下表现突出。本工作为生成模型提供了稳定且高保真的去学习范式,代码将公开发布。
原文摘要 · Abstract (English)
Text-to-image (T2I) diffusion models have achieved remarkable success in generating high-quality images from textual prompts. However, their ability to store vast amounts of knowledge raises concerns in scenarios where selective forgetting is necessary, such as removing copyrighted content, reducing biases, or eliminating harmful concepts. While existing unlearning methods can remove certain concepts, they struggle with multi-concept forgetting due to instability, residual knowledge persistence, and generation quality degradation. To address these challenges, we propose \textbf{Dynamic Mask coupled with Concept-Aware Loss}, a novel unlearning framework designed for multi-concept forgetting in diffusion models. Our \textbf{Dynamic Mask} mechanism adaptively updates gradient masks based on current optimization states, allowing selective weight modifications that prevent interference with unrelated knowledge. Additionally, our \textbf{Concept-Aware Loss} explicitly guides the unlearning process by enforcing semantic consistency through superclass alignment, while a regularization loss based on knowledge distillation ensures that previously unlearned concepts remain forgotten during sequential unlearning. We conduct extensive experiments to evaluate our approach. Results demonstrate that our method outperforms existing unlearning techniques in forgetting effectiveness, output fidelity, and semantic coherence, particularly in multi-concept scenarios. Our work provides a principled and flexible framework for stable and high-fidelity unlearning in generative models. The code will be released publicly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。