通过概念层级结构,高效批量擦除扩散模型中的多个敏感概念。
Mass Concept Erasure in Diffusion Models with Concept Hierarchy
- 构建父-子概念层级,将相关概念分组统一处理。
- 仅用一组参数即可同时擦除多个语义相关的概念。
- 提出新方法保持上位概念生成质量,适合安全可控生成场景。
扩散模型在生成内容时可能产生有害信息,现有概念擦除方法需为每个概念单独微调参数,随着擦除概念增多,效率下降且影响生成质量。本文提出一种超类型-子类型概念层级结构:将语义相关概念(如鹦鹉、秃鹰)归入同一上位概念(如鸟类)下,通过共享一组可学习参数实现联合擦除。擦除阶段采用标准扩散正则化以保留未掩码区域的去噪能力。为缓解因过度擦除子类型导致上位概念生成退化的问题,提出新型低秩适配方法SuPLoRA,将上位概念信息编码于冻结的下投影矩阵中,仅更新上投影矩阵。理论分析验证其有效性。构建更难的基准测试,要求同时擦除跨领域概念,涵盖名人、物体及色情内容。
原文摘要 · Abstract (English)
The success of diffusion models has raised concerns about the generation of unsafe or harmful content, prompting concept erasure approaches that fine-tune modules to suppress specific concepts while preserving general generative capabilities. However, as the number of erased concepts grows, these methods often become inefficient and ineffective, since each concept requires a separate set of fine-tuned parameters and may degrade the overall generation quality. In this work, we propose a supertype-subtype concept hierarchy that organizes erased concepts into a parent-child structure. Each erased concept is treated as a child node, and semantically related concepts (e.g., macaw, and bald eagle) are grouped under a shared parent node, referred to as a supertype concept (e.g., bird). Rather than erasing concepts individually, we introduce an effective and efficient group-wise suppression method, where semantically similar concepts are grouped and erased jointly by sharing a single set of learnable parameters. During the erasure phase, standard diffusion regularization is applied to preserve denoising process in unmasked regions. To mitigate the degradation of supertype generation caused by excessive erasure of semantically related subtypes, we propose a novel method called Supertype-Preserving Low-Rank Adaptation (SuPLoRA), which encodes the supertype concept information in the frozen down-projection matrix and updates only the up-projection matrix during erasure. Theoretical analysis demonstrates the effectiveness of SuPLoRA in mitigating generation performance degradation. We construct a more challenging benchmark that requires simultaneous erasure of concepts across diverse domains, including celebrities, objects, and pornographic content.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。