提出跨模型的图文联合越狱攻击,暴露多模态大模型安全漏洞。
Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
- 通过图文迭代交互生成通用越狱后缀和图像
- 在5个主流多模态模型上实现高成功率越狱
- 揭示现有安全机制对多模态攻击的严重不足
多模态大语言模型(MLLMs)融合视觉等多源信息,更接近人类智能。然而,文本越狱攻击已暴露其安全缺陷。本文提出统一的多模态通用越狱攻击框架,利用迭代图文交互与迁移策略,生成通用对抗性后缀与图像。实验评估了LLaVA、Yi-VL、MiniGPT4、MiniGPT-v2和InstructBLIP等模型,发现该攻击能有效触发高质量有害内容生成,暴露出多模态安全对齐的重大缺陷,表明当前防护机制难以应对复杂多模态威胁,亟需强化安全协议。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have evolved into Multimodal Large Language Models (MLLMs), significantly enhancing their capabilities by integrating visual information and other types, thus aligning more closely with the nature of human intelligence, which processes a variety of data forms beyond just text. Despite advancements, the undesirable generation of these models remains a critical concern, particularly due to vulnerabilities exposed by text-based jailbreak attacks, which have represented a significant threat by challenging existing safety protocols. Motivated by the unique security risks posed by the integration of new and old modalities for MLLMs, we propose a unified multimodal universal jailbreak attack framework that leverages iterative image-text interactions and transfer-based strategy to generate a universal adversarial suffix and image. Our work not only highlights the interaction of image-text modalities can be used as a critical vulnerability but also validates that multimodal universal jailbreak attacks can bring higher-quality undesirable generations across different MLLMs. We evaluate the undesirable context generation of MLLMs like LLaVA, Yi-VL, MiniGPT4, MiniGPT-v2, and InstructBLIP, and reveal significant multimodal safety alignment issues, highlighting the inadequacy of current safety mechanisms against sophisticated multimodal attacks. This study underscores the urgent need for robust safety measures in MLLMs, advocating for a comprehensive review and enhancement of security protocols to mitigate potential risks associated with multimodal capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。