让扩散模型持续删除概念时,不损伤附近概念,效果更稳定。
Locality-Aware Continual Unlearning for Diffusion Models
- 用模型自身判断最相似的提示词做精准删除目标
- 在删除概念周围保护相近概念,避免连带损坏
- 支持连续10次删除,适合真实场景长期使用
现实世界中文本到图像扩散模型需持续移除新出现的隐私、版权或安全相关概念。现有方法仅支持单步删除,连续执行3-5次后即崩溃。我们发现其不稳定性源于两点:(i) 删除目标选择粗糙,导致误差累积;(ii) 缺乏对语义邻近概念的局部保护,这些概念共享内部表示,首当其冲受损。由于损害集中在遗忘概念的局部语义邻域,仅靠全局回放无法避免。为此,我们提出局部感知持续遗忘(LACU),包含两个互补机制:局部感知目标选择根据得分预测距离,选取模型自身认为与原提示最相似的上下文保留提示,使每步更新尽可能小且精准;局部感知回放利用同一度量识别遗忘概念附近最接近的保留概念,并作为局部功能正则化回放,直接保护最脆弱邻域。结合教师-学生蒸馏与轻量级ℓ₂参数正则化,LACU可在10次连续步骤中保持稳定遗忘,显著优于近期基线,在相关保留(RR_acc)和通用保留(GR_acc)上均有提升。
原文摘要 · Abstract (English)
Real-world deployment of text-to-image diffusion models requires continual concept removal as new privacy, copyright, or safety obligations arise over time. Existing unlearning methods, however, are designed for single-step deletion and collapse after only 3-5 sequential applications. We trace this instability to two compounding factors: (i) coarse mapping targets that cause degradation to accumulate unnecessarily across steps, and (ii) the absence of local protection for semantically neighboring concepts, whose shared internal representations make them the first to suffer collateral damage. Because this damage is strongest in the local semantic neighborhood of the forget concept, global replay alone cannot prevent it. Building on this analysis, we propose Locality-Aware Continual Unlearning (LACU), a framework with two complementary mechanisms. Locality-Aware Target Selection chooses, for each forget prompt, the context-preserving mapping prompt that the diffusion model itself treats as most similar to the original prompt, measured by score-prediction distance (how differently the model denoises the same noisy image under two text conditions), ensuring each update is as small and targeted as possible. Locality-Aware Replay uses the same metric to identify the retain concepts closest to the forget concept in the model's own representation and replays them as a local functional regularizer, directly shielding the most vulnerable neighborhood. Combined with teacher-student distillation and lightweight $\ell_2$ parameter regularization, LACU maintains stable unlearning over 10 sequential steps, preserving significantly higher related retention ($RR_{\text{acc}}$) and general retention ($GR_{\text{acc}}$) than recent baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。