arXiv:2503.04856cs.CLcs.AI2025-03中稿 · ACL被引 12

将多轮对抗提示压缩为单轮,提升攻击效率与成功率。

M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs

  • 用三种方法将多轮对话转为结构化单轮提示。
  • 成功率最高达95.9%,比原方法高17.5个百分点。
  • 适合大规模模型安全测试,揭示防御漏洞。

我们提出一种新框架,将多轮对抗性‘越狱’提示合并为单轮查询,显著降低大型语言模型(LLMs)对抗测试的人工成本。尽管多轮人类越狱攻击已证明具有高成功率,但需大量人力和时间。我们的多轮转单轮(M2S)方法——连字符化、编号化、代码化——系统性地将多轮对话重构为结构化单轮提示。尽管去除了迭代交互,这些提示仍保持甚至增强攻击效力:在多轮人类越狱(MHJ)数据集上的评估显示,M2S方法在多个先进LLMs上实现70.6%至95.9%的攻击成功率。令人惊讶的是,单轮提示比原多轮攻击高出最多17.5个百分点,同时平均减少超过一半的令牌消耗。进一步分析表明,将恶意请求嵌入枚举或代码类结构中,可利用‘上下文盲区’,绕过原生防护机制和外部输入输出过滤器。通过将多轮对话转化为简洁单轮提示,M2S框架为大规模红队测试提供可扩展工具,并揭示了当前LLM防御体系的关键弱点。

原文摘要 · Abstract (English)

We introduce a novel framework for consolidating multi-turn adversarial ``jailbreak'' prompts into single-turn queries, significantly reducing the manual overhead required for adversarial testing of large language models (LLMs). While multi-turn human jailbreaks have been shown to yield high attack success rates, they demand considerable human effort and time. Our multi-turn-to-single-turn (M2S) methods -- Hyphenize, Numberize, and Pythonize -- systematically reformat multi-turn dialogues into structured single-turn prompts. Despite removing iterative back-and-forth interactions, these prompts preserve and often enhance adversarial potency: in extensive evaluations on the Multi-turn Human Jailbreak (MHJ) dataset, M2S methods achieve attack success rates from 70.6 percent to 95.9 percent across several state-of-the-art LLMs. Remarkably, the single-turn prompts outperform the original multi-turn attacks by as much as 17.5 percentage points while cutting token usage by more than half on average. Further analysis shows that embedding malicious requests in enumerated or code-like structures exploits ``contextual blindness'', bypassing both native guardrails and external input-output filters. By converting multi-turn conversations into concise single-turn prompts, the M2S framework provides a scalable tool for large-scale red teaming and reveals critical weaknesses in contemporary LLM defenses.

越狱攻击红队测试LLM安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。