让扩散模型删除特定概念,还能保持生成质量不下降。
TRACE: Trajectory-Constrained Concept Erasure in Diffusion Models
- 通过约束采样轨迹,只在后期去噪阶段消除目标概念。
- 在多个数据集上优于现有方法,对敏感内容删除效果更彻底。
- 兼容Stable Diffusion和FLUX等主流模型,适用性强。
文本到图像的扩散模型展现出前所未有的生成能力,但其可能生成色情内容、敏感身份或受版权保护的风格等有害概念,引发隐私、公平与安全问题。概念擦除旨在从生成模型中移除特定概念信息。本文提出一种新方法——轨迹约束注意力概念擦除(TRACE),可在保留整体生成质量的前提下,有效擦除扩散模型中的目标概念。该方法结合严谨的理论框架,给出概念可被证明抑制的条件,并设计适配传统潜在扩散模型(如Stable Diffusion)及新兴修正流模型(如FLUX)的微调方案。我们推导出跨注意力层的闭式更新公式,以消除目标概念的隐藏表示;并引入轨迹感知的微调目标,在采样后期引导去噪过程避开该概念,从而维持无关内容的生成保真度。实验在多个基准测试上进行,涵盖物体类别、名人面孔、艺术风格及来自I2P数据集的显性内容。结果表明,TRACE在删除效果与输出质量上均达到当前最优,显著优于ANT、EraseAnything和MACE等近期方法。
原文摘要 · Abstract (English)
Text-to-image diffusion models have shown unprecedented generative capability, but their ability to produce undesirable concepts (e.g.~pornographic content, sensitive identities, copyrighted styles) poses serious concerns for privacy, fairness, and safety. {Concept erasure} aims to remove or suppress specific concept information in a generative model. In this paper, we introduce \textbf{TRACE (Trajectory-Constrained Attentional Concept Erasure)}, a novel method to erase targeted concepts from diffusion models while preserving overall generative quality. Our approach combines a rigorous theoretical framework, establishing formal conditions under which a concept can be provably suppressed in the diffusion process, with an effective fine-tuning procedure compatible with both conventional latent diffusion (Stable Diffusion) and emerging rectified flow models (e.g.~FLUX). We first derive a closed-form update to the model's cross-attention layers that removes hidden representations of the target concept. We then introduce a trajectory-aware finetuning objective that steers the denoising process away from the concept only in the late sampling stages, thus maintaining the model's fidelity on unrelated content. Empirically, we evaluate TRACE on multiple benchmarks used in prior concept erasure studies (object classes, celebrity faces, artistic styles, and explicit content from the I2P dataset). TRACE achieves state-of-the-art performance, outperforming recent methods such as ANT, EraseAnything, and MACE in terms of removal efficacy and output quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。