让扩散模型精准删除特定概念,同时保留其他生成能力。
SPARE: Self-distillation for PARameter-Efficient Removal
- 分两阶段:先定位关键参数,再用自蒸馏替换目标概念。
- 在UnlearnCanvas上超越现有方法,实现高擦除率与强保留性。
- 适合需合规删数据的AI图像生成场景,如隐私保护。
机器遗忘旨在移除训练模型中特定数据或概念的影响,同时保持整体性能,这一能力日益受到数据保护法规和负责任AI实践的需求驱动。尽管已有进展,文本到图像扩散模型中的遗忘仍面临计算成本高、遗忘与保留难以平衡的挑战。我们提出自蒸馏参数高效删除方法SPARE,该方法结合参数定位与自蒸馏,分为两个阶段:第一阶段利用基于梯度的显著性识别最相关参数,并通过稀疏低秩适配器约束更新,实现轻量级、局部化修改;第二阶段应用自蒸馏目标,将目标概念替换为用户定义的替代概念,同时保留其他概念的行为。此外,我们设计了一种针对扩散模型的时间步采样策略,仅聚焦于特定概念的关键时间步,提升遗忘效率。SPARE在UnlearnCanvas基准上超越当前最优方法,多个数据集的消融实验表明其可精细调控遗忘-保留权衡。结果表明,SPARE在多种领域均实现了强概念擦除与高保留性,是基于扩散模型图像生成中选择性遗忘的可行解决方案。
原文摘要 · Abstract (English)
Machine Unlearning aims to remove the influence of specific data or concepts from trained models while preserving overall performance, a capability increasingly required by data protection regulations and responsible AI practices. Despite recent progress, unlearning in text-to-image diffusion models remains challenging due to high computational costs and the difficulty of balancing effective forgetting with retention of unrelated concepts. We introduce Self-distillation for PARameter Efficient Removal (SPARE), a two-stage unlearning method for image generation that combines parameter localization with self-distillation. SPARE first identifies parameters most responsible for generation of the unwanted concepts using gradient-based saliency and constrains updates through sparse low rank adapters, ensuring lightweight, localized modifications. In a second stage, SPARE applies a self-distillation objective that overwrites the unwanted concept with a user-defined surrogate while preserving behavior for other concepts. In addition we proposed a timestep sampling scheme for diffusion models to target only the crucial timesteps for a given concept leading to efficient unlearning. SPARE surpasses the current state-of-the-art on the UnlearnCanvas benchmark, and ablation studies on several datasets indicate fine-grained control over the forgetting-retention trade-off. Our results demonstrate that SPARE achieves strong concept erasure and high retainability across various domains, making it a suitable solution for selective unlearning in diffusion-based image generation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。