精准定位并替换扩散模型中的敏感概念，不破坏其他区域。
Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization
- 用少样本训练的局部定位器精准识别目标概念位置。
- 无需训练的双提示交叉注意力模块实现概念替换。
- 适合需要安全可控生成的图像编辑场景。
随着大规模扩散模型的发展，其在生成高质量图像方面表现出色，但常产生色情或暴力等不当内容。现有概念移除方法通常引导生成过程，却可能意外修改无关区域，导致与原始模型不一致。本文提出一种针对扩散模型中特定概念替换的新方法，可在不影响非目标区域的前提下，精准移除指定概念。该方法引入专用概念定位器，在去噪过程中精确识别目标概念，采用少样本学习，仅需少量标注数据即可训练。在定位区域内，设计了无需训练的双提示交叉注意力（DPCA）模块，实现目标概念的替换，最大限度减少对周围内容的影响。我们在概念定位精度和替换效率上进行评估，实验结果表明，该方法在目标概念定位上精度更高，替换过程保持内容连贯性，对非目标区域影响极小，优于现有方法。
原文摘要 · Abstract (English)
As large-scale diffusion models continue to advance, they excel at producing high-quality images but often generate unwanted content, such as sexually explicit or violent content. Existing methods for concept removal generally guide the image generation process but can unintentionally modify unrelated regions, leading to inconsistencies with the original model. We propose a novel approach for targeted concept replacing in diffusion models, enabling specific concepts to be removed without affecting non-target areas. Our method introduces a dedicated concept localizer for precisely identifying the target concept during the denoising process, trained with few-shot learning to require minimal labeled data. Within the identified region, we introduce a training-free Dual Prompts Cross-Attention (DPCA) module to substitute the target concept, ensuring minimal disruption to surrounding content. We evaluate our method on concept localization precision and replacement efficiency. Experimental results demonstrate that our method achieves superior precision in localizing target concepts and performs coherent concept replacement with minimal impact on non-target areas, outperforming existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。