用历史攻击经验提升大模型越狱效率,效果显著
Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
- 构建可动态更新的经验池,结构化存储过往越狱尝试
- 相比现有方法成功率提升17%,效率提高2.7倍
- 适合研究大模型安全漏洞与防御机制的开发者
大型语言模型(LLMs)在安全约束下生成符合人类对齐的内容。然而,当前已知的“越狱提示”技术可绕过安全机制,诱导模型输出恶意内容。研究越狱有助于发现LLM的安全漏洞并指导鲁棒安全框架的构建。为应对攻击模板随模型演进而过时的问题,现有方法采用迭代变异和动态优化以实现更自动化的越狱攻击。但这些方法面临效率低下和重复优化的挑战,因忽视了历史攻击经验的价值。为此,我们提出首个实现经验结构形式化表示的自动化越狱框架JailExpert,能够基于语义漂移对经验进行分组,并支持经验池的动态更新。大量实验表明,JailExpert显著提升了攻击的有效性与效率:相比当前最先进的黑盒越狱方法,平均攻击成功率提升17%,攻击效率提高2.7倍。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) generate human-aligned content under certain safety constraints. However, the current known technique ``jailbreak prompt'' can circumvent safety-aligned measures and induce LLMs to output malicious content. Research on Jailbreaking can help identify vulnerabilities in LLMs and guide the development of robust security frameworks. To circumvent the issue of attack templates becoming obsolete as models evolve, existing methods adopt iterative mutation and dynamic optimization to facilitate more automated jailbreak attacks. However, these methods face two challenges: inefficiency and repetitive optimization, as they overlook the value of past attack experiences. To better integrate past attack experiences to assist current jailbreak attempts, we propose the \textbf{JailExpert}, an automated jailbreak framework, which is the first to achieve a formal representation of experience structure, group experiences based on semantic drift, and support the dynamic updating of the experience pool. Extensive experiments demonstrate that JailExpert significantly improves both attack effectiveness and efficiency. Compared to the current state-of-the-art black-box jailbreak methods, JailExpert achieves an average increase of 17\% in attack success rate and 2.7 times improvement in attack efficiency. Our implementation is available at \href{https://github.com/xiZAIzai/JailExpert}{XiZaiZai/JailExpert}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。