混合攻击方法提升大模型越狱成功率,突破现有安全防护。
Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses
- 结合标记级与提示级攻击,互补提升越狱效果。
- 在Llama-3上攻击成功率达91.6%,高于单一方法。
- 可绕过梯度护甲等先进防御,适合研究安全漏洞者。
预训练语言模型(PTLMs)和大语言模型(LLMs)广泛应用,但其仍易受攻击,尤其在推理阶段存在标记级和提示级越狱威胁。标记级攻击通过对抗性序列转移至黑盒模型,但易被检测且依赖梯度优化;提示级攻击利用语义结构诱导有害响应,但依赖迭代反馈且不可靠。为解决二者局限,本文提出两种混合方法:GCG + PAIR 和 GCG + WordGame,分别在多个Vicuna与Llama模型上评估。GCG + PAIR 在未设防模型中持续提升成功率,例如在Llama-3上达到91.6%的攻击成功率(ASR),显著高于PAIR的58.4%基准。GCG + WordGame保持超80%的高ASR,即使在更严格的Mistral-Sorry-Bench评测下依然有效。关键的是,两类混合方法均保留迁移能力,并成功穿透梯度护甲(Gradient Cuff)与JBShield等高级防御机制,这些机制可完全阻挡单一模式攻击。结果揭示当前安全架构中未报告的漏洞,凸显原始成功率与防御鲁棒性间的权衡,强调需构建面向自适应对手的综合防护体系。
原文摘要 · Abstract (English)
The advancement of Pre-Trained Language Models (PTLMs) and Large Language Models (LLMs) has led to their widespread adoption across diverse applications. Despite their success, these models remain vulnerable to attacks that exploit their inherent weaknesses to bypass safety measures. Two primary inference-phase threats are token-level and prompt-level jailbreaks. Token-level attacks embed adversarial sequences that transfer well to black-box models like GPT but leave detectable patterns and rely on gradient-based token optimization, whereas prompt-level attacks use semantically structured inputs to elicit harmful responses yet depend on iterative feedback that can be unreliable. To address the complementary limitations of these methods, we propose two hybrid approaches that integrate token- and prompt-level techniques to enhance jailbreak effectiveness across diverse PTLMs. GCG + PAIR and the newly explored GCG + WordGame hybrids were evaluated across multiple Vicuna and Llama models. GCG + PAIR consistently raised attack-success rates over its constituent techniques on undefended models; for instance, on Llama-3, its Attack Success Rate (ASR) reached 91.6%, a substantial increase from PAIR's 58.4% baseline. Meanwhile, GCG + WordGame matched the raw performance of WordGame maintaining a high ASR of over 80% even under stricter evaluators like Mistral-Sorry-Bench. Crucially, both hybrids retained transferability and reliably pierced advanced defenses such as Gradient Cuff and JBShield, which fully blocked single-mode attacks. These findings expose previously unreported vulnerabilities in current safety stacks, highlight trade-offs between raw success and defensive robustness, and underscore the need for holistic safeguards against adaptive adversaries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。