攻击者用看似无害的工具链组合,让大模型代理在多轮操作中执行恶意任务。
STAC: When Innocent Tools Form Dangerous Chains for LLM Agents
- 设计自动闭环框架,生成隐蔽的多步工具调用链。
- 实测显示8个主流代理平均攻击成功率高达91.2%。
- 现有防御无效,需基于完整行为序列的推理来应对。
随着大模型向具备工具使用能力的自主代理发展,其安全挑战已超越传统内容安全范畴。本文提出一种新型多轮攻击框架——\\(STAC),利用代理的工具调用特性构造攻击链。该框架将看似无害的单个工具调用串联成整体有害操作,仅在最终执行时暴露风险。核心为自动化闭环流水线:合成可执行的多步工具链,通过环境内执行验证,并逆向生成能可靠触发恶意序列的隐蔽多轮提示。我们生成并系统评估了483个\\(STAC案例,涵盖1,352组用户-代理-环境交互,覆盖多种领域、任务、代理类型及10类失效模式。结果表明,当前顶尖大模型代理对\\(STAC高度脆弱,平均攻击成功率达91.2%,除一个代理外其余均超90%。防御分析发现,现有基于提示的防护效果有限。为此,我们提出一种基于推理的新型防御提示,在初始轮次中将攻击成功率降低最多28.8%;但面对自适应攻击,该优势迅速减弱。相比之下,基于经验的防御(ToolShield)在持续多轮交互中表现更持久。研究揭示关键缺口:防御工具化代理需对整个动作序列及其累积效应进行推理,而非孤立评估提示或响应。
原文摘要 · Abstract (English)
As LLMs advance into autonomous agents with tool-use capabilities, they introduce security challenges that extend beyond traditional content-based LLM safety concerns. This paper introduces Sequential Tool Attack Chaining (\STAC), a novel multi-turn attack framework that exploits agent tool use. \STAC chains together tool calls that each appear harmless in isolation but, when combined, collectively enable harmful operations that only become apparent at the final execution step. At the core of \STAC is an automated, closed-loop pipeline that synthesizes executable multi-step tool chains, validates them through in-environment execution, and reverse-engineers stealthy multi-turn prompts that reliably induce agents to execute the verified malicious sequence. Using this framework, we generate and systematically evaluate 483 \STAC cases, featuring 1,352 sets of user-agent-environment interactions and spanning diverse domains, tasks, agent types, and 10 failure modes. Our evaluations show that state-of-the-art LLM agents are highly vulnerable to \STAC, with an average final attack success rate (ASR) of 91.2\% -- exceeding 90\% for all but one of the eight agents evaluated. We further perform defense analysis and find that existing prompt-based defenses provide limited protection. To address this gap, we propose a new reasoning-driven defense prompt that achieves the strongest initial-turn protection, cutting ASR by up to 28.8\%; however, this advantage erodes sharply under adaptive attacks, and an experience-based defense (ToolShield) proves more durable over sustained multi-turn interactions. These results highlight a crucial gap: defending tool-enabled agents requires reasoning over entire action sequences and their cumulative effects, rather than evaluating isolated prompts or responses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。