让文生图模型持续移除特定概念,还能保持图文一致
Continuous Concepts Removal in Text-to-image Diffusion Models
- 用遗传算法生成提示词,配合知识蒸馏约束图文对齐
- 连续移除多个概念后仍能保持高质量图像生成
- 适合需要长期去除非必要内容的AI绘画应用
文生图扩散模型在从文本生成高质量图像方面表现出色,但存在侵犯版权或生成令人不适内容的风险。移除特定概念是应对这一问题的可行方案。然而,现有方法在需持续移除概念的现实场景中表现不佳,导致文本与生成图像对齐度下降。为此,我们提出一种新方法CCRT,采用设计的知识蒸馏范式,在连续概念移除过程中通过遗传算法生成的一组提示词,约束文本-图像对齐行为,该算法结合了定制的模糊策略。我们在多种概念移除任务上进行了广泛实验,结果表明,无论在算法指标还是人工评估中,CCRT均能有效实现目标概念的连续移除,同时保持模型的高生成质量(如文本-图像对齐)。
原文摘要 · Abstract (English)
Text-to-image diffusion models have shown an impressive ability to generate high-quality images from input textual descriptions. However, concerns have been raised about the potential for these models to create content that infringes on copyrights or depicts disturbing subject matter. Removing specific concepts from these models is a promising potential solution to this problem. However, existing methods for concept removal do not work well in practical but challenging scenarios where concepts need to be continuously removed. Specifically, these methods lead to poor alignment between the text prompts and the generated image after the continuous removal process. To address this issue, we propose a novel approach called CCRT that includes a designed knowledge distillation paradigm. It constrains the text-image alignment behavior during the continuous concept removal process by using a set of text prompts generated through our genetic algorithm, which employs a designed fuzzing strategy. We conduct extensive experiments involving the removal of various concepts. The results evaluated through both algorithmic metrics and human studies demonstrate that our CCRT can effectively remove the targeted concepts in a continuous manner while maintaining the high generation quality (e.g., text-image alignment) of the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。