用心理技巧自动生成大规模对话劫持数据,揭示大模型安全漏洞。
Automating Deception: Scalable Multi-Turn LLM Jailbreaks
- 将心理战术转化为可复现模板,自动生成多轮攻击数据
- GPT模型在有上下文时攻击成功率最高升32个百分点
- Gemini 2.5 Flash几乎免疫此类攻击,适合安全研究参考
多轮对话攻击利用如‘脚踏门槛’(Foot-in-the-Door, FITD)等心理学原理,通过小请求逐步诱导大语言模型(LLM)突破安全对齐,构成持续威胁。现有防御进展受限于人工构建数据集,难以规模化。本文提出一种新型自动化流水线,可生成大规模、基于心理机制的多轮越狱数据集。我们将FITD技术系统化为可复现模板,构建了包含1500个场景的基准数据集,覆盖非法行为与冒犯性内容。我们在七种来自三大主流模型家族的模型上,评估其在多轮(含历史)与单轮(无历史)条件下的表现。结果显示,上下文鲁棒性差异显著:GPT系列模型在有对话历史时攻击成功率(ASR)最高提升32个百分点;而Google Gemini 2.5 Flash表现出极强韧性,几乎不受影响;Anthropic Claude 3 Haiku则具较强但非完全的抵抗能力。这些发现凸显当前安全架构在处理对话上下文上的根本分歧,强调需发展抵御叙事操控的防御机制。
原文摘要 · Abstract (English)
Multi-turn conversational attacks, which leverage psychological principles like Foot-in-the-Door (FITD), where a small initial request paves the way for a more significant one, to bypass safety alignments, pose a persistent threat to Large Language Models (LLMs). Progress in defending against these attacks is hindered by a reliance on manual, hard-to-scale dataset creation. This paper introduces a novel, automated pipeline for generating large-scale, psychologically-grounded multi-turn jailbreak datasets. We systematically operationalize FITD techniques into reproducible templates, creating a benchmark of 1,500 scenarios across illegal activities and offensive content. We evaluate seven models from three major LLM families under both multi-turn (with history) and single-turn (without history) conditions. Our results reveal stark differences in contextual robustness: models in the GPT family demonstrate a significant vulnerability to conversational history, with Attack Success Rates (ASR) increasing by as much as 32 percentage points. In contrast, Google's Gemini 2.5 Flash exhibits exceptional resilience, proving nearly immune to these attacks, while Anthropic's Claude 3 Haiku shows strong but imperfect resistance. These findings highlight a critical divergence in how current safety architectures handle conversational context and underscore the need for defenses that can resist narrative-based manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。