用概率框架生成隐蔽的图文对抗样本，突破视觉语言模型安全防线
VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models
- 将图文攻击建模为联合后验分布学习，实现协同对抗输入生成
- 在GPT-4o上比最佳基线提升53.75%攻击成功率，跨模型稳定有效
- 适合安全研究者、红队测试人员，用于发现多模态模型深层漏洞
视觉语言模型（VLMs）虽拓展了大语言模型的视觉推理能力，但其多模态结构也引入了未充分探索的安全漏洞。现有多模态红队方法多依赖脆弱模板，局限于单次攻击场景，且仅暴露有限漏洞。为此，本文提出VERA-V，一种基于变分推断的框架，将多模态越狱发现重构为对图文提示联合后验分布的学习。该概率视角支持生成隐蔽的耦合对抗输入，绕过模型防护机制。我们训练一个轻量级攻击者以近似后验分布，实现高效采样多样越狱样本，并提供漏洞分布洞察。VERA-V融合三种互补策略：(i) 基于排版的文本提示嵌入有害线索，(ii) 基于扩散的图像合成引入对抗信号，(iii) 结构化干扰项打散VLM注意力。在HarmBench和HADES基准上的实验表明，VERA-V持续优于现有最优基线，在开源与前沿VLM上均表现卓越，对GPT-4o的攻击成功率较最佳基线最高提升53.75%。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) extend large language models with visual reasoning, but their multimodal design also introduces new, underexplored vulnerabilities. Existing multimodal red-teaming methods largely rely on brittle templates, focus on single-attack settings, and expose only a narrow subset of vulnerabilities. To address these limitations, we introduce VERA-V, a variational inference framework that recasts multimodal jailbreak discovery as learning a joint posterior distribution over paired text-image prompts. This probabilistic view enables the generation of stealthy, coupled adversarial inputs that bypass model guardrails. We train a lightweight attacker to approximate the posterior, allowing efficient sampling of diverse jailbreaks and providing distributional insights into vulnerabilities. VERA-V further integrates three complementary strategies: (i) typography-based text prompts that embed harmful cues, (ii) diffusion-based image synthesis that introduces adversarial signals, and (iii) structured distractors to fragment VLM attention. Experiments on HarmBench and HADES benchmarks show that VERA-V consistently outperforms state-of-the-art baselines on both open-source and frontier VLMs, achieving up to 53.75% higher attack success rate (ASR) over the best baseline on GPT-4o. We include the code on the project page available here: https://github.com/kxwhiowo/VERA-V
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。