通过操控注意力提升大模型越狱攻击效果
AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation
- 基于注意力机制优化越狱攻击策略
- 在Llama-2上提升7%，Gemma上提升10%
- 适用于未知攻击目标和黑盒模型，解释性更强
本文研究基于Transformer的大语言模型（LLMs）对越狱攻击的脆弱性，聚焦于基于优化的贪婪坐标梯度（GCG）策略。我们发现攻击效果与模型内部行为呈正相关：当模型更关注安全对齐的系统提示时，攻击效果下降。基于此，提出一种通过操纵注意力分数来增强越狱攻击的方法——AttnGCG。实验表明，AttnGCG在多种大模型上均显著提升攻击有效性，使Llama-2系列平均提升约7%，Gemma系列提升约10%。该方法还表现出强迁移能力，可有效攻击未见的有害目标及GPT-3.5、GPT-4等黑盒模型。此外，注意力分数可视化更具可解释性，有助于理解攻击机制。代码已开源：https://github.com/UCSC-VLAA/AttnGCG-attack。
原文摘要 · Abstract (English)
This paper studies the vulnerabilities of transformer-based Large Language Models (LLMs) to jailbreaking attacks, focusing specifically on the optimization-based Greedy Coordinate Gradient (GCG) strategy. We first observe a positive correlation between the effectiveness of attacks and the internal behaviors of the models. For instance, attacks tend to be less effective when models pay more attention to system prompts designed to ensure LLM safety alignment. Building on this discovery, we introduce an enhanced method that manipulates models' attention scores to facilitate LLM jailbreaking, which we term AttnGCG. Empirically, AttnGCG shows consistent improvements in attack efficacy across diverse LLMs, achieving an average increase of ~7% in the Llama-2 series and ~10% in the Gemma series. Our strategy also demonstrates robust attack transferability against both unseen harmful goals and black-box LLMs like GPT-3.5 and GPT-4. Moreover, we note our attention-score visualization is more interpretable, allowing us to gain better insights into how our targeted attention manipulation facilitates more effective jailbreaking. We release the code at https://github.com/UCSC-VLAA/AttnGCG-attack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。