无需训练即可快速删除扩散模型中的特定概念,且不损伤其他功能。
CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models
- 通过正交投影直接修改模型权重,实现概念删除
- 2秒内完成编辑,对无关内容影响极小
- 适合需要精准去除特定内容的AI图像生成场景
随着文本到图像模型的发展,生成不安全、侵权或侵犯隐私内容的风险也在增加。现有安全干预手段如数据清洗、微调、推理时过滤等,常面临概念删除不彻底、易被绕过、计算效率低或损害无关能力等问题。本文提出CURE,一种无需训练的概念遗忘框架,直接在预训练扩散模型的权重空间操作,实现快速、可解释且高度精确的不良概念抑制。核心是谱消融器(Spectral Eraser),基于奇异值分解识别与待遗忘/保留概念相关的特征子空间,通过闭式正交投影分离出特定概念特征并予以消除。该算子单步更新即可生成去除非目标概念的模型,无需重新训练、监督或迭代优化。为平衡去毒效果与泛化能力,引入谱正则化扩展机制,按奇异向量重要性选择性调节遗忘强度。全过程均为闭式解,仅需2秒即可完成擦除。在多个基准测试中,CURE相比先前方法更高效彻底地移除了特定艺术风格、物体、身份或敏感内容,对原始生成能力损伤小,并展现出更强的抗对抗攻击鲁棒性。
原文摘要 · Abstract (English)
As Text-to-Image models continue to evolve, so does the risk of generating unsafe, copyrighted, or privacy-violating content. Existing safety interventions - ranging from training data curation and model fine-tuning to inference-time filtering and guidance - often suffer from incomplete concept removal, susceptibility to jail-breaking, computational inefficiency, or collateral damage to unrelated capabilities. In this paper, we introduce CURE, a training-free concept unlearning framework that operates directly in the weight space of pre-trained diffusion models, enabling fast, interpretable, and highly specific suppression of undesired concepts. At the core of our method is the Spectral Eraser, a closed-form, orthogonal projection module that identifies discriminative subspaces using Singular Value Decomposition over token embeddings associated with the concepts to forget and retain. Intuitively, the Spectral Eraser identifies and isolates features unique to the undesired concept while preserving safe attributes. This operator is then applied in a single step update to yield an edited model in which the target concept is effectively unlearned - without retraining, supervision, or iterative optimization. To balance the trade-off between filtering toxicity and preserving unrelated concepts, we further introduce an Expansion Mechanism for spectral regularization which selectively modulates singular vectors based on their relative significance to control the strength of forgetting. All the processes above are in closed-form, guaranteeing extremely efficient erasure in only $2$ seconds. Benchmarking against prior approaches, CURE achieves a more efficient and thorough removal for targeted artistic styles, objects, identities, or explicit content, with minor damage to original generation ability and demonstrates enhanced robustness against red-teaming.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。