攻击者可让视觉语言模型误判图像内容，且攻击可跨模型通用。
Transferable Adversarial Attacks on Black-Box Vision-Language Models
- 用可迁移的扰动干扰图像理解，诱导错误解读。
- 对GPT-4o、Claude等主流模型攻击成功率超80%。
- 适合研究安全漏洞或对抗防御的开发者参考。
视觉大语言模型（VLLMs）在图文混合输入场景中日益普及。尽管已有研究证明文本和纯视觉任务中存在可迁移的对抗攻击，但针对VLLMs的此类威胁仍不明确。本文首次系统性分析表明，目标型对抗样本能高效攻击GPT-4o、Claude、Gemini等主流私有模型。攻击者可构造扰动，使模型将危险内容误判为安全，忽略敏感信息，或生成符合攻击者意图的错误回答。进一步发现，通用扰动可在多类图像上一致引发模型误判。在物体识别、视觉问答和图像描述任务上的实验表明，该漏洞广泛存在于当前顶尖模型中，凸显了提升VLLM安全性的紧迫性。
原文摘要 · Abstract (English)
Vision Large Language Models (VLLMs) are increasingly deployed to offer advanced capabilities on inputs comprising both text and images. While prior research has shown that adversarial attacks can transfer from open-source to proprietary black-box models in text-only and vision-only contexts, the extent and effectiveness of such vulnerabilities remain underexplored for VLLMs. We present a comprehensive analysis demonstrating that targeted adversarial examples are highly transferable to widely-used proprietary VLLMs such as GPT-4o, Claude, and Gemini. We show that attackers can craft perturbations to induce specific attacker-chosen interpretations of visual information, such as misinterpreting hazardous content as safe, overlooking sensitive or restricted material, or generating detailed incorrect responses aligned with the attacker's intent. Furthermore, we discover that universal perturbations -- modifications applicable to a wide set of images -- can consistently induce these misinterpretations across multiple proprietary VLLMs. Our experimental results on object recognition, visual question answering, and image captioning show that this vulnerability is common across current state-of-the-art models, and underscore an urgent need for robust mitigations to ensure the safe and secure deployment of VLLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。