针对扩散模型的对抗攻击,提出通用净化方法提升鲁棒性
Universal Adversarial Purification with DDIM Metric Loss for Stable Diffusion
- 利用DDIM反演中干净与对抗图像的重建差异优化净化过程
- 在多种攻击下保持高净化效果,且适配不同SD版本和文本提示
- 动态调整训练轮次,提升效率不牺牲质量,适合实际部署
Stable Diffusion(SD)在训练数据含对抗噪声时常产生劣化输出。现有净化方法主要针对分类任务,难以应对针对VAE编码器、UNet去噪器或两者联合的SD特有攻击策略。为此,本文提出面向扩散模型的通用对抗净化框架UDAP。UDAP利用干净图像与对抗图像在去噪扩散隐式模型(DDIM)反演中的重建行为差异,通过最小化DDIM度量损失实现有效净化。此外,引入基于重建误差的动态周期调整策略,在不降低净化质量的前提下显著提升效率。实验表明,UDAP对多种攻击(如针对VAE的PID、针对UNet的Anti-DreamBooth、混合型MIST,以及增强版Anti-DF、MetaCloak)均具强鲁棒性,并在不同SD版本与文本提示间具有良好泛化能力,具备实际应用潜力。
原文摘要 · Abstract (English)
Stable Diffusion (SD) often produces degraded outputs when the training dataset contains adversarial noise. Adversarial purification offers a promising solution by removing adversarial noise from contaminated data. However, existing purification methods are primarily designed for classification tasks and fail to address SD-specific adversarial strategies, such as attacks targeting the VAE encoder, UNet denoiser, or both. To address the gap in SD security, we propose Universal Diffusion Adversarial Purification (UDAP), a novel framework tailored for defending adversarial attacks targeting SD models. UDAP leverages the distinct reconstruction behaviors of clean and adversarial images during Denoising Diffusion Implicit Models (DDIM) inversion to optimize the purification process. By minimizing the DDIM metric loss, UDAP can effectively remove adversarial noise. Additionally, we introduce a dynamic epoch adjustment strategy that adapts optimization iterations based on reconstruction errors, significantly improving efficiency without sacrificing purification quality. Experiments demonstrate UDAP's robustness against diverse adversarial methods, including PID (VAE-targeted), Anti-DreamBooth (UNet-targeted), MIST (hybrid), and robustness-enhanced variants like Anti-Diffusion (Anti-DF) and MetaCloak. UDAP also generalizes well across SD versions and text prompts, showcasing its practical applicability in real-world scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。