用通用扰动保护图像,防扩散模型篡改。
Universal Image Immunization against Diffusion-based Image Editing via Semantic Injection
- 设计通用对抗扰动,一扰动护所有图像。
- 在小扰动下阻断编辑,优于定制化方法。
- 跨模型通用性强,适合实际部署。
扩散模型的进展使文本引导图像编辑成为可能,但也带来了深度伪造和未经授权使用等伦理与法律风险。为应对这些风险,基于对抗攻击的图像免疫防御应运而生。然而,现有方法大多需要针对每张图像优化或推理时引入额外神经网络,限制了可扩展性与实用性。本文提出首个基于通用对抗扰动(UAP)的图像免疫框架,生成单一、图像无关的对抗扰动,专为扩散式编辑流程设计。受目标攻击中UAP启发,该方法旨在生成一个能诱导扩散模型将输入图像误认为特定语义目标的扰动,同时压制原始内容,误导模型注意力,从而通过覆盖原语义有效阻止未经授权的编辑。大量实验表明,作为首个通用免疫方法,本方案在UAP设置下显著优于多个基线。值得注意的是,尽管通用扰动本身难度高,本方法在更受限的扰动预算下仍达到与图像特异性方法相当甚至更优的性能,并展现出强大的黑盒跨模型迁移能力。
原文摘要 · Abstract (English)
Diffusion model advances have enabled powerful text-guided image editing, but also raise ethical and legal risks such as deepfakes and unauthorized use. To prevent these risks, adversarial attack-based image immunization has emerged as a promising defense against AI-driven semantic manipulation. Yet, most existing approaches require image-specific optimization or additional neural networks at inference time, hindering scalability and practicality. In this paper, we propose the first universal adversarial perturbation-based image immunization framework that generates a single, image-agnostic adversarial perturbation specifically designed for diffusion-based editing pipelines. Inspired by UAP used in targeted attacks, our method aims to generate a UAP that induces diffusion models to misinterpret the input image as a specific semantic target. Simultaneously, it suppresses original content to misdirect the model's attention during editing, thereby effectively blocking unauthorized edits by overwriting the image's original semantics via the UAP. Extensive experiments show that our method, as the first universal immunization approach, significantly outperforms several baselines in the UAP setting. Notably, despite the inherent difficulty of universal perturbations, our method achieves competitive or superior performance compared to image-specific methods under a more restricted perturbation budget, while also exhibiting strong black-box transferability across diverse diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。