arXiv:2508.05755cs.CVcs.AI2025-08被引 6

用动态引导机制让扩散模型精准删去特定概念,不伤整体画质。

UnGuide: Learning to Forget with LoRA-Guided Diffusion Models

  • 基于LoRA与动态引导控制删除过程,只影响目标概念。
  • 在图像去除非相关物体和敏感内容任务中效果优于现有方法。
  • 适合需要安全可控生成的场景,如内容过滤或隐私保护。

大规模文本到图像扩散模型的兴起引发了对其潜在滥用的担忧,尤其在生成有害或误导性内容方面。这凸显了机器遗忘的迫切需求——即在不损害整体性能的前提下,从预训练模型中移除特定知识或概念。一种可行方案是低秩适应(LoRA),它能高效地微调模型实现针对性遗忘。然而,LoRA常无意中改变无关内容,导致图像保真度和真实感下降。为此,我们提出UnGuide——一种新方法,引入动态推理机制UnGuidance,利用无分类器引导(CFG)对遗忘过程进行精确控制。UnGuide根据去噪初期几步的稳定性动态调节引导系数:包含被删除概念的提示中,LoRA模块主导并被基础模型抵消;无关提示则由基础模型主导,保持内容保真度。实验证明,UnGuide实现了可控的概念移除,同时保留了扩散模型的表现力,在物体擦除和显式内容删除任务中均优于现有基于LoRA的方法。

原文摘要 · Abstract (English)

Recent advances in large-scale text-to-image diffusion models have heightened concerns about their potential misuse, especially in generating harmful or misleading content. This underscores the urgent need for effective machine unlearning, i.e., removing specific knowledge or concepts from pretrained models without compromising overall performance. One possible approach is Low-Rank Adaptation (LoRA), which offers an efficient means to fine-tune models for targeted unlearning. However, LoRA often inadvertently alters unrelated content, leading to diminished image fidelity and realism. To address this limitation, we introduce UnGuide -- a novel approach which incorporates UnGuidance, a dynamic inference mechanism that leverages Classifier-Free Guidance (CFG) to exert precise control over the unlearning process. UnGuide modulates the guidance scale based on the stability of a few first steps of denoising processes, enabling selective unlearning by LoRA adapter. For prompts containing the erased concept, the LoRA module predominates and is counterbalanced by the base model; for unrelated prompts, the base model governs generation, preserving content fidelity. Empirical results demonstrate that UnGuide achieves controlled concept removal and retains the expressive power of diffusion models, outperforming existing LoRA-based methods in both object erasure and explicit content removal tasks.

扩散模型机器遗忘LoRA内容安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。