通过串行提示链让大模型忽略恶意指令,实现单次查询高成功率越狱攻击。
SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains
- 将恶意提示藏于多个正常提示中,利用模型注意力偏差绕过检测。
- 单次查询攻击成功率显著高于现有方法,对开源与闭源模型均有效。
- 揭示模型在复杂提示结构下的安全漏洞,适合安全研究者关注。
随着大语言模型(LLMs)在各类应用中的广泛集成,其被滥用的风险也日益增加,引发重大安全担忧。已有多种越狱攻击被提出以评估LLM的安全防御能力。当前攻击主要依赖场景伪装、提示混淆、提示优化及迭代优化来隐藏恶意内容。特别地,单个查询中的串行提示链可能导致模型只关注部分提示而忽略其他,从而便于上下文操控。本文提出SequentialBreak,一种新型越狱攻击,利用此漏洞。我们探讨了包括题库、对话补全和游戏环境在内的多种场景,其中恶意提示被嵌入于看似正常的提示序列中,诱使模型生成有害响应。这些场景的叙事结构表明SequentialBreak能灵活适配多种提示格式。大量实验表明,SequentialBreak仅用一次查询即显著提升攻击成功率,优于现有基线方法,适用于开源与闭源模型。本研究凸显了亟需更强大、更具鲁棒性的防护机制以增强LLM安全性并防止潜在滥用。所有结果文件及关联网站见:https://anonymous.4open.science/r/JailBreakAttack-4F3B/
原文摘要 · Abstract (English)
As the integration of the Large Language Models (LLMs) into various applications increases, so does their susceptibility to misuse, raising significant security concerns. Numerous jailbreak attacks have been proposed to assess the security defense of LLMs. Current jailbreak attacks mainly rely on scenario camouflage, prompt obfuscation, prompt optimization, and prompt iterative optimization to conceal malicious prompts. In particular, sequential prompt chains in a single query can lead LLMs to focus on certain prompts while ignoring others, facilitating context manipulation. This paper introduces SequentialBreak, a novel jailbreak attack that exploits this vulnerability. We discuss several scenarios, not limited to examples like Question Bank, Dialog Completion, and Game Environment, where the harmful prompt is embedded within benign ones that can fool LLMs into generating harmful responses. The distinct narrative structures of these scenarios show that SequentialBreak is flexible enough to adapt to various prompt formats beyond those discussed. Extensive experiments demonstrate that SequentialBreak uses only a single query to achieve a substantial gain of attack success rate over existing baselines against both open-source and closed-source models. Through our research, we highlight the urgent need for more robust and resilient safeguards to enhance LLM security and prevent potential misuse. All the result files and website associated with this research are available in this GitHub repository: https://anonymous.4open.science/r/JailBreakAttack-4F3B/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。