arXiv:2503.09446cs.CVcs.AI2025-03被引 8

用稀疏自编码器实现零样本概念删除,不降生成质量

Sparse Autoencoder as a Zero-Shot Classifier for Concept Erasing in Text-to-Image Diffusion Models

  • 先用稀疏自编码器解析概念由多个特征组成
  • 关闭目标概念对应特征,实现精准删除且不影响正常生成
  • 无需再训练即可删多个概念,适合内容安全场景

文本到图像扩散模型虽能生成高质量图像,但可能产生有害或误导性内容。现有去除非期望概念的方法常导致正常生成性能下降。本文提出 Interpret then Deactivate(ItD)框架,通过稀疏自编码器(SAE)将每个概念解释为多个特征的组合,永久关闭与目标概念相关的特征,从而将 SAE 重用于零样本分类,判断输入提示是否含目标概念,实现扩散模型中选择性概念删除。此外,ItD 可无需额外训练扩展至多概念删除。在名人身份、艺术风格和敏感内容等任务上的实验证明,ItD 能有效消除目标概念,同时保持正常生成性能。该方法对试图绕过内容过滤的对抗性提示也具备鲁棒性。代码已开源。

原文摘要 · Abstract (English)

Text-to-image (T2I) diffusion models have achieved remarkable progress in generating high-quality images but also raise people's concerns about generating harmful or misleading content. While extensive approaches have been proposed to erase unwanted concepts without requiring retraining from scratch, they inadvertently degrade performance on normal generation tasks. In this work, we propose Interpret then Deactivate (ItD), a novel framework to enable precise concept removal in T2I diffusion models while preserving overall performance. ItD first employs a sparse autoencoder (SAE) to interpret each concept as a combination of multiple features. By permanently deactivating the specific features associated with target concepts, we repurpose SAE as a zero-shot classifier that identifies whether the input prompt includes target concepts, allowing selective concept erasure in diffusion models. Moreover, we demonstrate that ItD can be easily extended to erase multiple concepts without requiring further training. Comprehensive experiments across celebrity identities, artistic styles, and explicit content demonstrate ItD's effectiveness in eliminating targeted concepts without interfering with normal concept generation. Additionally, ItD is also robust against adversarial prompts designed to circumvent content filters. Code is available at: https://github.com/NANSirun/Interpret-then-deactivate.

文本生成扩散模型概念删除零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。