提出LURE方法,让被删除的概念在扩散模型中重新出现
LURE: Latent Space Unblocking for Multi-Concept Reawakening in Diffusion Models
- 从潜在空间重构入手,通过语义再绑定恢复被切断的图文关联
- 可同时高保真唤醒多个被删除概念,且在多种擦除方法下有效
- 适合研究模型安全性和概念可控性的研究人员参考
概念擦除旨在抑制扩散模型中的敏感内容,但近期研究表明,被擦除的概念仍可能被重新唤醒,暴露出擦除方法的漏洞。现有重唤醒方法主要依赖提示词优化来操控采样轨迹,忽视了其他生成因素,限制了对内在机制的全面理解。本文将生成过程建模为隐式函数,实现对文本条件、模型参数和潜在状态等多因素的理论分析。理论上证明,扰动任一因素均可唤醒被擦除概念。基于此,提出一种新方法:潜在空间解封(LURE),通过重建潜在空间并引导采样轨迹实现概念重唤醒。具体而言,其语义再绑定机制通过将去噪预测与目标分布对齐,恢复被切断的文本-视觉关联。但在多概念场景中,直接重建会导致梯度冲突与特征纠缠。为此,引入梯度场正交化,强制特征正交以避免相互干扰。此外,潜在语义识别引导采样(LSIS)通过后验密度验证确保重唤醒过程的稳定性。大量实验表明,LURE可在多种擦除任务与方法下,实现多个被擦除概念的同时、高保真重唤醒。
原文摘要 · Abstract (English)
Concept erasure aims to suppress sensitive content in diffusion models, but recent studies show that erased concepts can still be reawakened, revealing vulnerabilities in erasure methods. Existing reawakening methods mainly rely on prompt-level optimization to manipulate sampling trajectories, neglecting other generative factors, which limits a comprehensive understanding of the underlying dynamics. In this paper, we model the generation process as an implicit function to enable a comprehensive theoretical analysis of multiple factors, including text conditions, model parameters, and latent states. We theoretically show that perturbing each factor can reawaken erased concepts. Building on this insight, we propose a novel concept reawakening method: Latent space Unblocking for concept REawakening (LURE), which reawakens erased concepts by reconstructing the latent space and guiding the sampling trajectory. Specifically, our semantic re-binding mechanism reconstructs the latent space by aligning denoising predictions with target distributions to reestablish severed text-visual associations. However, in multi-concept scenarios, naive reconstruction can cause gradient conflicts and feature entanglement. To address this, we introduce Gradient Field Orthogonalization, which enforces feature orthogonality to prevent mutual interference. Additionally, our Latent Semantic Identification-Guided Sampling (LSIS) ensures stability of the reawakening process via posterior density verification. Extensive experiments demonstrate that LURE enables simultaneous, high-fidelity reawakening of multiple erased concepts across diverse erasure tasks and methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。