arXiv:2601.10971cs.CRcs.CL2026-01被引 1

AJAR让红队攻击可调用、可回滚,提升大模型越狱成功率。

AJAR: Adaptive Jailbreak Architecture for Red-teaming

  • 将多轮越狱算法封装为可调用服务,支持回滚与策略切换
  • 在HarmBench上将越狱成功率从65.0%提升至76.0%
  • 适合研究大模型安全与智能体攻击的开发者使用

大型语言模型安全评估正从内容过滤转向行为安全,因现代系统具备持久状态、工具访问和自主控制能力。现有越狱框架在自适应多轮攻击与智能体运行时之间存在断层:攻击算法多为独立脚本,而智能体平台通常缺乏显式回滚、工具模拟或策略切换接口。我们提出AJAR,一个将多轮越狱算法作为可调用MCP服务暴露的红队框架,并由审计代理在基于Petri的工具感知运行时中协调执行。AJAR整合了Crescendo、ActorAttack和X-Teaming三种代表性攻击,统一提供规划、提示生成、优化、评估和上下文控制接口。在200个HarmBench验证行为上,AJAR将X-Teaming的攻击成功率达从65.0%提升至76.0%,累计成功率提前一回合达到80%;对Crescendo的复现效果优于PyRIT(91.0% vs. 87.5%)。行为级分析显示,性能提升集中在高难度类别,且依赖回滚修复对话记录。进一步发现,工具接入并非均等扩展攻击面:ActorAttack成功率从51.0%升至56.0%,而Crescendo从91.0%降至78.0%,X-Teaming从76.0%降至55.5%,尤其依赖长语义累积的类别下降最明显。结果表明,AJAR是评估真实智能体约束下多轮越狱的有效基础。代码与数据见https://github.com/douyipu/ajar。

原文摘要 · Abstract (English)

Large language model (LLM) safety evaluation is moving from content moderation to action security as modern systems gain persistent state, tool access, and autonomous control loops. Existing jailbreak frameworks still leave a gap between adaptive multi-turn attacks and agentic runtimes: attack algorithms are usually packaged as monolithic scripts, while agent harnesses rarely expose explicit abstractions for rollback, tool simulation, or strategy switching. We present AJAR, a red-teaming framework that exposes multi-turn jailbreak algorithms as callable MCP services and lets an Auditor Agent orchestrate them inside a tool-aware runtime built on Petri. AJAR integrates three representative attacks, namely Crescendo, ActorAttack, and X-Teaming, under a shared service interface for planning, prompt generation, optimization, evaluation, and context control. On 200 HarmBench validation behaviors, AJAR improves X-Teaming from 65.0% to 76.0% attack success rate (ASR), reaches 80% cumulative success one turn earlier than the native implementation, and reproduces Crescendo more effectively than PyRIT (91.0% vs. 87.5% ASR). Behavior-level analysis shows that these gains are concentrated in hard categories and frequently depend on rollback-enabled transcript repair. We further show that tool access reshapes rather than uniformly enlarges the attack surface: ActorAttack rises from 51.0% to 56.0% ASR with tools, whereas Crescendo drops from 91.0% to 78.0% and X-Teaming from 76.0% to 55.5%, with the sharpest declines appearing in categories that rely on long semantic buildup. These results position AJAR as a practical foundation for evaluating multi-turn jailbreaks under realistic agent constraints. Code and data are available at https://github.com/douyipu/ajar.

越狱攻击智能体安全红队测试多轮攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。