提出抗微调的去毒框架,让生成模型更安全。
Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuning
- 将微调建模为隐式优化问题,用梯度估计抑制有害行为恢复。
- 在多种数据集和微调方式下,安全保留率显著优于现有方法。
- 适合关注生成模型安全性的研究者与工业应用开发者。
文本到图像扩散模型在生成质量上取得显著进展,并被广泛用于个性化应用。然而,这些模型常继承有毒预训练数据中的不安全行为,引发日益严重的安全问题。尽管近期的安全驱动去毒方法在抑制模型毒性方面取得进展,但其对下游微调极为脆弱——我们发现,即使在完全无害的数据集上微调,当前最优方法也基本丧失有效性。为此,本文提出ResAlign,一种具备更强抗微调韧性的安全驱动去毒框架。通过将下游微调建模为基于Moreau包络重构的隐式优化问题,ResAlign实现了高效梯度估计,以最小化有害行为的恢复。此外,引入元学习策略模拟多样化的微调场景,提升泛化能力。在多个数据集、微调方法与配置下的大量实验表明,ResAlign在持续保持安全性的同时,有效保留良性生成能力,显著优于已有去毒方法。代码与预训练模型已开源:https://github.com/AntigoneRandy/ResAlign。
原文摘要 · Abstract (English)
Text-to-image (T2I) diffusion models have achieved impressive image generation quality and are increasingly fine-tuned for personalized applications. However, these models often inherit unsafe behaviors from toxic pretraining data, raising growing safety concerns. While recent safety-driven unlearning methods have made promising progress in suppressing model toxicity, they are found to be fragile to downstream fine-tuning, as we reveal that state-of-the-art methods largely fail to retain their effectiveness even when fine-tuned on entirely benign datasets. To mitigate this problem, in this paper, we propose ResAlign, a safety-driven unlearning framework with enhanced resilience against downstream fine-tuning. By modeling downstream fine-tuning as an implicit optimization problem with a Moreau envelope-based reformulation, ResAlign enables efficient gradient estimation to minimize the recovery of harmful behaviors. Additionally, a meta-learning strategy is proposed to simulate a diverse distribution of fine-tuning scenarios to improve generalization. Extensive experiments across a wide range of datasets, fine-tuning methods, and configurations demonstrate that ResAlign consistently outperforms prior unlearning approaches in retaining safety, while effectively preserving benign generation capability. Our code and pretrained models are publicly available at https://github.com/AntigoneRandy/ResAlign.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。