通过稀疏注意力机制，高效擦除扩散模型中的特定概念。
Empty SPACE: Cross-Attention Sparsity for Concept Erasure in Diffusion Models
- 用闭式更新迭代稀疏化交叉注意力参数，实现概念擦除。
- 在大模型上提升擦除效果，达到80%-90%注意力稀疏度。
- 适合需要快速、低存储成本擦除敏感内容的研究者。
从文本到图像的扩散模型中擦除特定概念，对避免生成版权或不当内容至关重要。闭式概念擦除方法相比基于反向传播的技术更快速，但在从Stable Diffusion 1.5等小模型扩展到Stable Diffusion XL等大模型时效果下降。为此，我们提出基于稀疏交叉注意力的概念擦除方法SPACE。SPACE通过闭式更新迭代修改模型的交叉注意力参数，同时引入稀疏性并擦除目标概念。通过将概念映射集中于低维子空间，SPACE在擦除效果上优于密集基线。大量实验表明其在擦除有效性与对抗提示鲁棒性方面均有提升。此外，该方法可实现80%-90%的交叉注意力稀疏度，使修改后参数的存储需求降低70%，兼具高效与内存友好特性。
原文摘要 · Abstract (English)
Erasing specific concepts from text-to-image diffusion models is essential for avoiding the generation of copyrighted and explicit content. Closed-form concept erasure methods offer a fast alternative to backpropagation-based techniques, but they become less effective when scaling from smaller models such as Stable Diffusion 1.5 to larger models like Stable Diffusion XL. To maintain erasure effectiveness in these larger-scale architectures, we propose SParse cross-Attention-based Concept Erasure (SPACE). SPACE iteratively modifies the cross-attention parameters of a model with a closed-form update that jointly induces sparsity and erases target concepts. By concentrating the concept mapping to a lower-dimensional subspace, SPACE achieves superior erasure efficacy compared to dense baselines. Extensive experimental results show improvements in erasure effectiveness and robustness against adversarial prompts. Furthermore, SPACE achieves 80\%-90\% cross-attention sparsity, reducing the storage requirements for saving the modified parameters by 70\%, demonstrating its memory efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。