用进化算法自动发现更强的单轮越狱模板。
X-Teaming Evolutionary M2S: Automated Discovery of Multi-turn to Single-turn Jailbreak Templates
- 通过语言模型引导演化，自动搜索优化越狱模板。
- 在GPT-4.1上达到44.8%成功率（103/230），发现两个新模板族。
- 适合安全研究者与对抗样本探索者参考。
多轮到单轮（M2S）将迭代红队测试压缩为单一结构化提示，但先前工作依赖少量人工编写的模板。本文提出X-Teaming Evolutionary M2S，一个基于语言模型引导演化的自动化框架，用于发现和优化M2S模板。系统结合12个来源的智能采样与受StrongREJECT启发的LLM作为裁判，并记录可审计日志。通过设定成功阈值θ=0.70维持选择压力，历经五代进化，获得两个新模板家族，在GPT-4.1上实现44.8%的整体成功率（103/230）。2,500次跨模型测试（裁判固定）表明结构优势可迁移，但效果因目标模型而异；有两个模型在同一阈值下得分为零。还发现提示长度与得分呈正相关，支持长度感知评估。结果证明结构级搜索是可复现的强单轮探测路径，强调阈值校准与跨模型评估的重要性。代码、配置与数据集已公开于https://github.com/hyunjun1121/M2S-x-teaming。
原文摘要 · Abstract (English)
Multi-turn-to-single-turn (M2S) compresses iterative red-teaming into one structured prompt, but prior work relied on a handful of manually written templates. We present X-Teaming Evolutionary M2S, an automated framework that discovers and optimizes M2S templates through language-model-guided evolution. The system pairs smart sampling from 12 sources with an LLM-as-judge inspired by StrongREJECT and records fully auditable logs. Maintaining selection pressure by setting the success threshold to $θ= 0.70$, we obtain five evolutionary generations, two new template families, and 44.8% overall success (103/230) on GPT-4.1. A balanced cross-model panel of 2,500 trials (judge fixed) shows that structural gains transfer but vary by target; two models score zero at the same threshold. We also find a positive coupling between prompt length and score, motivating length-aware judging. Our results demonstrate that structure-level search is a reproducible route to stronger single-turn probes and underscore the importance of threshold calibration and cross-model evaluation. Code, configurations, and artifacts are available at https://github.com/hyunjun1121/M2S-x-teaming.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。