用注意力机制自动生成稳定锚点,实现更可靠的内容删减。
AutoAnchor: Stable Diffusion Unlearning Using Cross-Attention as a Manifold Surrogate

- 通过交叉注意力构建流形近似锚点,避免人工选择偏差。
- 在多个基线模型上提升内容移除效果31.04%,保留非目标图像质量4.18%。
- 可无缝集成现有方法,平均提升概念移除与通用性6%以上。
扩散模型中的内容删减对防止生成有害或侵权内容至关重要。现有方法或依赖人工语义锚点(易引入偏差),或采用无锚方法(导致潜在空间更新不稳定)。本文从流形假设出发,理论证明缺乏流形邻近锚点会引发显著法向漂移,降低删减性能。为此提出 mysys,一种两阶段框架,自动合成流形邻近锚点。针对直接几何优化计算不可行的问题,设计新颖的交叉注意力一致性损失,作为高效流形接近性的代理。实验表明, mysys 在多种先进基线上实现鲁棒且无偏的删减,显著提升目标概念移除能力(最高达31.04%的CLIP得分提升)与非目标内容保真度(最高4.18%提升)。同时可轻松集成至现有方法,平均提升概念移除6.30%、通用性6.65%。
原文摘要 · Abstract (English)
Diffusion unlearning is essential for mitigating the generation of harmful or copyrighted content in text-to-image models. Current diffusion unlearning techniques determine the model update direction by either using alternatives of the target concept as an anchor or using empty prompts. The anchor-based method relies on manually and semantically-chosen anchors that risk biased unlearning, while the anchor-free method inherently suffers from unrobust unlearning due to unconstrained latent updates. In this work, we theoretically formalize such unstable diffusion unlearning issues under the manifold hypothesis and prove that lacking a manifold-proximal anchor inevitably induces significant normal-space drift that degrades unlearning performance. To achieve stable unlearning, we propose \mysysn, a two-stage framework that automatically synthesizes manifold-proximal anchors. However, direct geometric manifold optimization is computationally intractable. To address this challenge, \mysys introduces a novel cross-attention consistency loss which serves as a highly efficient surrogate of manifold proximity. Experimental results demonstrate that \mysys effectively achieves robust and unbiased unlearning across various state-of-the-art baselines, significantly improving targeted concept removal (by up to 31.04\% in CLIP score) and non-target utility (by up to 4.18\% in CLIP score). Moreover, \mysys can also be easily integrated into existing diffusion unlearning methods to enhance their unlearning performance (by 6.30\% for concept removal and 6.65\% for utility on average).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。