新框架让大模型安全测试更真实，能发现部分和完整危害意图的区别。
AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation
- 设计分阶段的多轮攻击流程，用语义信号驱动攻击进化
- 在6个前沿模型上实现高达97.6%的攻击成功率，严格标准下仍达66.7%-78.6%
- 通过多角色评审和检查清单，精准区分部分与完整可操作的危害输出
大型语言模型的安全评估长期依赖单轮攻击数据集和单一评分机制，低估了自适应多轮攻击的风险，并仅报告单一成功率，无法区分部分可执行输出与具备完整操作细节的威胁。我们提出AMT-X（自适应多轮利用），一个分阶段结构化的多轮红队测试框架。不同于以往依赖随意升级或自由规划的多轮攻击，AMT-X将攻击建模为由目标模型语义反馈驱动的显式、可复现的多阶段状态机，并以多角色评审团结合阶段条件检查清单替代单一评分，仅对具备可操作危害的输出判定成功。在六个前沿受害者模型（默认安全对齐，未添加额外监管层）及七个审核子类别中，AMT-X在宽松阈值下取得97.6%-100%的整体攻击成功率，但在要求完整、真实且可操作细节的严格门限下，成功率降至66.7%-78.6%，二者差距最高达33个百分点，凸显部分与完全可执行危害之间的显著差异。
原文摘要 · Abstract (English)
Safety evaluation of large language models (LLMs) relies largely on single-turn attack datasets and single-judge scoring, underestimating risk from adaptive multi-turn adversaries and reporting a single success rate that does not separate partially actionable outputs from those carrying complete operational detail. We propose AMT-X (Adaptive Multi-Turn Exploitation), a phase-structured multi-turn red-teaming framework. Unlike prior multi-turn attacks that rely on ad hoc escalation or free-form per-goal plans, AMT-X casts the attack as an explicit, reproducible multi-phase state machine driven by semantic signals from the victim, and replaces single-judge scoring with a multi-role jury whose phase-conditioned checklists gate success on actionable harm. Across six frontier victim models (queried under their default safety alignment, without added moderation layers) and seven Moderation sub-categories, AMT-X attains overall attack success rates of 97.6-100% under a lenient score threshold, but 66.7-78.6% under a stricter gate requiring complete, real, and operational detail: a gap of up to 33 percentage points between partially and fully actionable harm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。