扰动保护图像反而让扩散模型更听从文字指令,可能加剧风险
Is Perturbation-Based Image Protection Disruptive to Image Editing?
- 在图像中添加不可察觉的扰动以阻止编辑
- 多数情况下扰动后图像仍能生成符合提示的优质结果
- 适合关注生成安全与图像版权的从业者参考
当前先进的扩散模型(如Stable Diffusion)虽具强大图像生成能力,但也被滥用于传播虚假信息和抄袭版权内容。为降低图像编辑风险,现有保护方法通过向图像添加不可察觉的扰动来干扰扩散模型编辑。理想保护应使编辑输出为与原图无关的噪声图像。然而我们在多个领域(自然场景与艺术作品)及任务(图像到图像生成、风格编辑)的实验发现,此类保护并未完全实现目标。大多数情况下,受扰动图像仍能生成符合文本提示的优质输出。这表明,添加噪声可能在生成过程中意外增强图像与文本提示的关联性,导致意想不到的优化效果。因此,我们认为基于扰动的方法不足以提供对扩散模型编辑的可靠防护。
原文摘要 · Abstract (English)
The remarkable image generation capabilities of state-of-the-art diffusion models, such as Stable Diffusion, can also be misused to spread misinformation and plagiarize copyrighted materials. To mitigate the potential risks associated with image editing, current image protection methods rely on adding imperceptible perturbations to images to obstruct diffusion-based editing. A fully successful protection for an image implies that the output of editing attempts is an undesirable, noisy image which is completely unrelated to the reference image. In our experiments with various perturbation-based image protection methods across multiple domains (natural scene images and artworks) and editing tasks (image-to-image generation and style editing), we discover that such protection does not achieve this goal completely. In most scenarios, diffusion-based editing of protected images generates a desirable output image which adheres precisely to the guidance prompt. Our findings suggest that adding noise to images may paradoxically increase their association with given text prompts during the generation process, leading to unintended consequences such as better resultant edits. Hence, we argue that perturbation-based methods may not provide a sufficient solution for robust image protection against diffusion-based editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。