用树搜索模拟逐步突破大模型安全防线，实现高效多轮越狱。
Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Search
- 通过广度优先生成多条对抗性提示，利用前轮响应中的微小漏洞
- 在单次运行中对GPT-3.5-turbo成功率100%，GPT-4达97%
- 揭示安全机制如何逐轮退化，适合安全测试与模型评估者使用
我们提出Tempest，一种基于树搜索的多轮对抗框架，从渐进式侵蚀视角建模大语言模型（LLM）安全性的衰减过程。与依赖单一精心设计提示的单轮越狱不同，Tempest在每一轮对话中以广度优先方式生成多个对抗性提示，利用前一轮响应中部分合规输出所暴露的漏洞，并将其重新注入后续查询。通过追踪这些累积性的策略泄露，该方法揭示了微小让步如何演变为完全违规输出。在JailbreakBench数据集上的评估显示，Tempest在单次多轮运行中对GPT-3.5-turbo实现了100%成功率，对GPT-4达到97%，且所需查询次数少于Crescendo或GOAT等基线方法。该树搜索方法为理解模型安全机制如何随对话轮次逐步失效提供了深入洞察，强调了对语言模型进行鲁棒多轮测试的紧迫性。
原文摘要 · Abstract (English)
We introduce Tempest, a multi-turn adversarial framework that models the gradual erosion of Large Language Model (LLM) safety through a tree search perspective. Unlike single-turn jailbreaks that rely on one meticulously engineered prompt, Tempest expands the conversation at each turn in a breadth-first fashion, branching out multiple adversarial prompts that exploit partial compliance from previous responses. By tracking these incremental policy leaks and re-injecting them into subsequent queries, Tempest reveals how minor concessions can accumulate into fully disallowed outputs. Evaluations on the JailbreakBench dataset show that Tempest achieves a 100% success rate on GPT-3.5-turbo and 97% on GPT-4 in a single multi-turn run, using fewer queries than baselines such as Crescendo or GOAT. This tree search methodology offers an in-depth view of how model safeguards degrade over successive dialogue turns, underscoring the urgency of robust multi-turn testing procedures for language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。