用自动图文欺骗文本图像编辑，让修改失效却看不出异常。
Immunizing Images from Text to Image Editing via Adversarial Cross-Attention
- 用源图自动生成的描述干扰文本与图像的跨注意力对齐。
- 攻击使编辑效果大幅下降，但图像外观几乎无变化。
- 提出新评估方法，更真实反映攻击对内容一致性的影响。
近期基于文本的图像编辑技术实现了由自然语言引导的精细视觉操作，但易受对抗攻击。本文提出一种针对编辑方法视觉组件的新攻击——注意力攻击：通过使用源图像的自动生成描述作为伪造编辑提示，破坏文本提示与图像视觉表征间的跨注意力对齐，无需了解编辑模型或原始提示。为更可靠评估免疫效果，提出两种新评估策略：图文相似度（量化原图与对抗编辑间语义一致性）和语义交并比（IoU），通过分割掩码衡量空间布局破坏程度。在TEDBench++基准上的实验表明，该攻击显著降低编辑性能且人眼难以察觉。
原文摘要 · Abstract (English)
Recent advances in text-based image editing have enabled fine-grained manipulation of visual content guided by natural language. However, such methods are susceptible to adversarial attacks. In this work, we propose a novel attack that targets the visual component of editing methods. We introduce Attention Attack, which disrupts the cross-attention between a textual prompt and the visual representation of the image by using an automatically generated caption of the source image as a proxy for the edit prompt. This breaks the alignment between the contents of the image and their textual description, without requiring knowledge of the editing method or the editing prompt. Reflecting on the reliability of existing metrics for immunization success, we propose two novel evaluation strategies: Caption Similarity, which quantifies semantic consistency between original and adversarial edits, and semantic Intersection over Union (IoU), which measures spatial layout disruption via segmentation masks. Experiments conducted on the TEDBench++ benchmark demonstrate that our attack significantly degrades editing performance while remaining imperceptible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。