通过分层注意力均衡，让每一步攻击都有效提升成功率。
Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization
- 根据多层注意力分布选择性接受优化结果，避免无效步骤。
- 在三款模型上攻击成功率超80%，比现有方法高20%以上。
- 减少迭代次数，降低计算成本，适合安全测试场景使用。
在大型视觉语言模型（LVLM）领域，对抗性越狱攻击是一种红队测试方法，用于发现模型及其防御机制的安全漏洞。然而，我们发现一个关键问题：并非每个优化步骤都能带来积极效果，盲目接受每一步的优化结果可能降低整体攻击成功率。为此，我们提出一种名为HKVE（分层键值均衡）的新型越狱框架，基于不同层注意力分数的分布，有选择地接受梯度优化结果，确保每一步优化都对攻击产生正向贡献。大量实验表明，HKVE在MiniGPT4、LLaVA和Qwen-VL上分别达到75.08%、85.84%和81.00%的攻击成功率，显著优于现有方法，提升幅度分别为20.43%、21.01%和26.43%。此外，使每一步都有效不仅提升了攻击成功率，还减少了迭代次数，降低了计算开销。警告：本文包含潜在有害示例数据。
原文摘要 · Abstract (English)
In the realm of large vision-language models (LVLMs), adversarial jailbreak attacks serve as a red-teaming approach to identify safety vulnerabilities of these models and their associated defense mechanisms. However, we identify a critical limitation: not every adversarial optimization step leads to a positive outcome, and indiscriminately accepting optimization results at each step may reduce the overall attack success rate. To address this challenge, we introduce HKVE (Hierarchical Key-Value Equalization), an innovative jailbreaking framework that selectively accepts gradient optimization results based on the distribution of attention scores across different layers, ensuring that every optimization step positively contributes to the attack. Extensive experiments demonstrate HKVE's significant effectiveness, achieving attack success rates of 75.08% on MiniGPT4, 85.84% on LLaVA and 81.00% on Qwen-VL, substantially outperforming existing methods by margins of 20.43\%, 21.01\% and 26.43\% respectively. Furthermore, making every step effective not only leads to an increase in attack success rate but also allows for a reduction in the number of iterations, thereby lowering computational costs. Warning: This paper contains potentially harmful example data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。