揭示视觉攻击中干扰与精准注入的差异，发现真实有效攻击极少。
VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Models
- 区分模型输出被扰动和目标概念被准确注入两个维度，提出双轴评估方法。
- 在6615组测试中，仅0.756%实现非空注入，0.030%实现完全复现。
- 攻击效果集中在截图或文档类图像，提示文本转录是关键载体。
针对对齐多模态大语言模型的通用对抗攻击，现有研究报告攻击成功率在60%-80%，暗示视觉模态极易受不可察觉扰动影响。我们指出该数据混淆了两类事件：(i) 模型输出被扰动（影响），(ii) 攻击者指定的目标概念被实际输出（精准注入）。本文结合通用对抗攻击与AnyAttack，在$ L_{inf} $预算为16/255下构建双轴评估体系：使用确定性Ratcliff-Obershelp漂移分数衡量影响（程序基准），并引入四层级有序分类（无/弱/部分/确认）评估精准注入。评测采用DeepSeek-V4-Pro（思维模式），与Claude Opus 4.7校准，注入轴上Cohen's $κ$ = 0.77（显著一致性）。完整4475条输入哈希缓存随数据集发布，确保评审可逐比特复现结果。在覆盖6615对样本、四个开源VLM、七个攻击提示及七张测试图像的实验中，两轴差异达约90倍：66.4%样本被程序判定为扰动（LLM判断46.6%处于实质性或完全层级），但仅有0.756%（50/6615）进入非零注入层级，仅0.030%（2/6615）实现原文复现。成功注入集中于截图或文档类载体，其语义天然支持文本转录。BLIP-2在$ L_{inf} $=16/255下所有2205对均未显示可检测漂移，即使作为第一阶段代理亦然。数据集已发布于huggingface.co/datasets/jeffliulab/visinject，包含21张通用图像、147张对抗照片、6615个响应对、v3双轴评分结果及缓存。
原文摘要 · Abstract (English)
Universal adversarial attacks on aligned multimodal large language models are increasingly reported with attack success rates in the 60-80% range, suggesting the visual modality is highly vulnerable to imperceptible perturbations as a prompt-injection channel. We argue that this number conflates two distinct events: (i) the model's output was perturbed (Influence), and (ii) the attacker's chosen target concept was actually emitted (Precise Injection). We compose two existing techniques -- Universal Adversarial Attack and AnyAttack -- under an $L_{inf}$ budget of 16/255, and we add a dual-axis evaluation: a deterministic Ratcliff-Obershelp drift score for Influence (programmatic baseline) plus a 4-tier ordinal categorical none/weak/partial/confirmed for Precise Injection. The judge is DeepSeek-V4-Pro in thinking mode, calibrated against Claude Opus 4.7 with Cohen's $κ$ = 0.77 on the injection axis (substantial agreement); the entire 4475-entry SHA-256 input cache ships with the dataset so reviewers can re-derive paper numbers bit-exact without an API key. Across 6615 pairs over four open VLMs, seven attack prompts, and seven test images, the two axes diverge by roughly 90$\times$: 66.4% of pairs are programmatically disturbed (LLM-judged 46.6% at the substantial-or-complete tier), but only 0.756% (50/6615) reach any non-none injection tier and only 0.030% (2/6615) verbatim. The few injections that do land cluster on screenshot- or document-style carriers whose semantics already invite text transcription. BLIP-2 shows \emph{zero detectable drift} at $L_{inf}$ = 16/255 across all 2205 pairs even when used as a Stage-1 surrogate. We release the full dataset -- 21 universal images, 147 adversarial photos, 6,615 response pairs, the v3 dual-axis judge results, and the cache at huggingface.co/datasets/jeffliulab/visinject.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。