一张合成图可骗过多种多模态大模型的安全机制。
Universal Adversarial Attack on Aligned Multimodal LLMs
- 用反向传播生成通用对抗图像，触发模型输出指定内容。
- 在多个模型上攻击成功率最高达81%，超越纯文本攻击。
- 跨模型通用性强，适合研究安全漏洞与防御机制者阅读。
我们提出一种针对多模态大语言模型的通用对抗攻击方法，通过单张优化过的图像即可绕过多种查询和多个模型的安全对齐机制。该方法通过视觉编码器与语言头部的反向传播，生成一张合成图像，迫使模型输出目标短语（如“当然，给你”）或不安全内容，即使面对有害提示也有效。在SafeBench和MM-SafetyBench基准测试中，该方法攻击成功率高于现有基线，包括纯文本通用提示（某些模型最高达81%）。我们进一步通过同时训练多个多模态大模型验证了跨模型通用性。此外，一种多答案变体可生成更自然但依然恶意的响应。这些发现揭示了当前多模态对齐机制的关键漏洞，呼吁更强的对抗防御措施。代码与数据集将按Apache-2.0许可发布。警告：本文部分生成内容可能具有冒犯性。
原文摘要 · Abstract (English)
We propose a universal adversarial attack on multimodal Large Language Models (LLMs) that leverages a single optimized image to override alignment safeguards across diverse queries and even multiple models. By backpropagating through the vision encoder and language head, we craft a synthetic image that forces the model to respond with a targeted phrase (e.g., "Sure, here it is") or otherwise unsafe content -- even for harmful prompts. In experiments on the SafeBench and MM-SafetyBench benchmarks, our method achieves higher attack success rates than existing baselines, including text-only universal prompts (e.g., up to 81% on certain models). We further demonstrate cross-model universality by training on several multimodal LLMs simultaneously. Additionally, a multi-answer variant of our approach produces more natural-sounding (yet still malicious) responses. These findings underscore critical vulnerabilities in current multimodal alignment and call for more robust adversarial defenses. We will release code and datasets under the Apache-2.0 license. Warning: some content generated by Multimodal LLMs in this paper may be offensive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。