通过图文协同消除,精准移除不良内容且不影响其他正常生成。
One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework
- 用文本和对应图像共同描述目标概念,实现图文联合消去。
- 在不破坏其他正常概念的前提下,显著提升消除效果。
- 适合需要高精度内容安全控制的研究与应用者。
概念消除已成为防止文本到图像扩散模型生成不希望出现或有害内容的有效方法。然而,现有方法严重依赖人工设计的文本提示,难以在保证高消除效果的同时最小化对其他良性概念的影响。本文指出问题根源在于文本与图像模态间的固有差异,导致文本提示中的复杂概念知识难以有效传递至图像生成过程。为此,我们提出首个图文协同概念消除(Co-Erasing)框架,通过直接引入视觉监督,将目标概念同时由文本提示和由其诱导的不良图像共同描述,并利用负向引导降低目标概念的生成概率。该方法有效绕过文本与图像之间的知识鸿沟,显著提升消除效率。此外,我们设计了文本引导的图像概念精炼策略,使模型聚焦于与指定文本概念最相关的视觉特征,最大限度减少对其他良性概念的干扰。大量实验表明,Co-Erasing 显著优于现有先进方法,在消除效能与可用性之间实现了更优平衡。代码已开源:https://github.com/Ferry-Li/Co-Erasing。
原文摘要 · Abstract (English)
Concept erasing has recently emerged as an effective paradigm to prevent text-to-image diffusion models from generating visually undesirable or even harmful content. However, current removal methods heavily rely on manually crafted text prompts, making it challenging to achieve a high erasure (efficacy) while minimizing the impact on other benign concepts (usability). In this paper, we attribute the limitations to the inherent gap between the text and image modalities, which makes it hard to transfer the intricately entangled concept knowledge from text prompts to the image generation process. To address this, we propose a novel solution by directly integrating visual supervision into the erasure process, introducing the first text-image Collaborative Concept Erasing (Co-Erasing) framework. Specifically, Co-Erasing describes the concept jointly by text prompts and the corresponding undesirable images induced by the prompts, and then reduces the generating probability of the target concept through negative guidance. This approach effectively bypasses the knowledge gap between text and image, significantly enhancing erasure efficacy. Additionally, we design a text-guided image concept refinement strategy that directs the model to focus on visual features most relevant to the specified text concept, minimizing disruption to other benign concepts. Finally, comprehensive experiments suggest that Co-Erasing outperforms state-of-the-art erasure approaches significantly with a better trade-off between efficacy and usability. Codes are available at https://github.com/Ferry-Li/Co-Erasing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。