微调扩散模型可能意外复活被删除的概念，存在安全风险。
Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models
- 在模型微调时，已删除的概念会意外重新出现。
- 实验显示，对Stable Diffusion v1.4/v2.1微调后，曾被遗忘的内容重现。
- 提醒开发者警惕增量更新的不稳定性，尤其涉及版权或敏感内容时。
文本到图像扩散模型依赖海量网络数据进行训练。由于从头训练成本过高，开发者常采用增量更新策略，即结合微调（学习新概念）与“遗忘”（移除旧概念，如版权内容或敏感信息）。本文揭示了一种此前未知的关键漏洞：即使在非对抗性条件下，仅对看似无关的图像进行微调，也可能导致先前已被“遗忘”的概念意外“复现”。我们通过一系列实验，在Stable Diffusion v1.4和v2.1上系统研究了该现象——概念复现的成因与范围。结果表明，增量更新组合存在严重脆弱性，对当前确保生成模型安全与对齐的方法提出严峻挑战。
原文摘要 · Abstract (English)
Text-to-image diffusion models rely on massive, web-scale datasets. Training them from scratch is computationally expensive, and as a result, developers often prefer to make incremental updates to existing models. These updates often compose fine-tuning steps (to learn new concepts or improve model performance) with "unlearning" steps (to "forget" existing concepts, such as copyrighted works or explicit content). In this work, we demonstrate a critical and previously unknown vulnerability that arises in this paradigm: even under benign, non-adversarial conditions, fine-tuning a text-to-image diffusion model on seemingly unrelated images can cause it to "relearn" concepts that were previously "unlearned." We comprehensively investigate the causes and scope of this phenomenon, which we term concept resurgence, by performing a series of experiments which compose "concept unlearning" with subsequent fine-tuning of Stable Diffusion v1.4 and Stable Diffusion v2.1. Our findings underscore the fragility of composing incremental model updates, and raise serious new concerns about current approaches to ensuring the safety and alignment of text-to-image diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。