arXiv:2410.05694cs.CV2024-10

通过早期扩散过程的对抗噪声,有效防御文本引导图像编辑攻击。

DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing

  • 针对扩散模型早期阶段设计对抗噪声,提升防御效率。
  • 在多种掩码下仍保持强鲁棒性,且计算成本更低。
  • 适合关注隐私保护与内容安全的研究者和开发者。

近期扩散模型的发展推动了文本引导图像编辑的新范式,用户仅需简单文本提示即可生成逼真的编辑图像。然而,这类技术可能被滥用于制造误导或有害内容。尽管已有防御策略通过引入不可察觉的对抗噪声使模型失效,但对带掩码的复杂编辑仍无效。本文提出 DiffusionGuard,一种针对扩散模型图像编辑的鲁棒防御方法。通过分析模型机制,我们设计了一种新目标函数,生成针对扩散过程早期阶段的对抗噪声,显著提升噪声效率与有效性。同时引入掩码增强技术,提升测试时对各类掩码的鲁棒性。此外,构建了一个综合性基准,用于评估真实场景下对隐私威胁的防护能力。大量实验表明,该方法在保护强度、掩码鲁棒性上优于最强基线,且计算开销更低。同时具备更强迁移性和对去噪技术的更好抵抗能力。代码已公开于 https://github.com/choi403/DiffusionGuard。

原文摘要 · Abstract (English)

Recent advances in diffusion models have introduced a new era of text-guided image manipulation, enabling users to create realistic edited images with simple textual prompts. However, there is significant concern about the potential misuse of these methods, especially in creating misleading or harmful content. Although recent defense strategies, which introduce imperceptible adversarial noise to induce model failure, have shown promise, they remain ineffective against more sophisticated manipulations, such as editing with a mask. In this work, we propose DiffusionGuard, a robust and effective defense method against unauthorized edits by diffusion-based image editing models, even in challenging setups. Through a detailed analysis of these models, we introduce a novel objective that generates adversarial noise targeting the early stage of the diffusion process. This approach significantly improves the efficiency and effectiveness of adversarial noises. We also introduce a mask-augmentation technique to enhance robustness against various masks during test time. Finally, we introduce a comprehensive benchmark designed to evaluate the effectiveness and robustness of methods in protecting against privacy threats in realistic scenarios. Through extensive experiments, we show that our method achieves stronger protection and improved mask robustness with lower computational costs compared to the strongest baseline. Additionally, our method exhibits superior transferability and better resilience to noise removal techniques compared to all baseline methods. Our source code is publicly available at https://github.com/choi403/DiffusionGuard.

图像安全扩散模型对抗防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。