研究工具调用型智能体的隐蔽注入攻击，发现首次工具返回最危险
Depth-Dependent Indirect Prompt Injection in Tool-Calling ReAct Agents: Injection Depth, Payload Framing, and Turn-Budget Sensitivity
- 测试不同工具调用阶段注入指令的效果，定位安全薄弱点
- 首次工具返回时成功率60%，深度4以上基本失效
- 适合关注AI系统安全的开发者和评测人员
将思维链推理与工具调用结合的ReAct智能体正广泛应用于日程安排、文件检索等真实任务。其工具观测循环构成直接攻击面：攻击者若控制任一工具返回值，可嵌入指令引导智能体偏离用户目标，即间接提示注入攻击。现有基准在固定注入位置与条件下评估攻击成功率（ASR），未探索三个风险维度：注入深度、内容表述方式（框架）及允许交互轮次（轮次上限）。本文在20个场景、5类攻击上开展四组受控实验，共460次测试，使用GPT-4o-mini和Claude Haiku，API总成本低于0.36美元。实验1显示，GPT-4o-mini在深度1时ASR为60%，深度4和5降至0%（Cramer's V = 0.58, p < 0.001；仅限序列内深度1-3：V = 0.47, p = 0.0013），由模型初始抵抗与任务提前完成导致。实验2在Claude Haiku上复现，其在所有深度均实现0% ASR，源于保守调用策略与真实指令抵抗能力。实验3表明，在深度1时，中性框架下ASR为25%，角色化框架下达75%，跨度50个百分点，但样本量小（每条件N=20）未达统计显著。实验4确认，轮次上限为3、5、7时ASR稳定，说明轮次预算在此设置下非风险因素。结果确立注入深度为关键变量，并指出仅清理首条工具观测即可捕获67%的已测注入成功案例。
原文摘要 · Abstract (English)
ReAct agents that interleave chain-of-thought reasoning with tool calls are increasingly deployed for real tasks such as scheduling, file retrieval, and data access. Their tool observation loop creates a direct attack surface: an adversary who controls any tool's return value can embed instructions that redirect the agent away from the user's goal, a threat known as indirect prompt injection. Existing benchmarks evaluate attack success rate (ASR) at a fixed injection position under fixed conditions, leaving three risk dimensions unexplored: where in the tool sequence the payload appears (injection depth), what rhetorical register it uses (framing), and how many turns the agent is permitted (turn cap). We conduct four controlled studies on 20 scenarios spanning five attack categories, totalling 460 trials against GPT-4o-mini and Claude Haiku at a combined API cost under 0.36 USD. Study 1 shows that ASR against GPT-4o-mini decays from 60% at depth 1 to 0% at depths 4 and 5 (Cramer's V = 0.58, p < 0.001; restricted to within-sequence depths 1-3: V = 0.47, p = 0.0013), driven by model resistance at depth 1 and task completion before payload encounter at deeper positions. Study 2 replicates the depth experiment on Claude Haiku, which achieves 0% ASR at every depth through a combination of conservative tool invocation and genuine instruction resistance. Study 3 shows that framing modulates ASR between 25% (neutral) and 75% (persona) at depth 1, a 50-percentage-point range that does not reach statistical significance at N = 20 per condition. Study 4 confirms that ASR is stable across turn caps of 3, 5, and 7, indicating the turn budget is not a risk factor in this setting. Our results establish injection depth as the dominant variable and show that sanitising only the first tool observation captures 67% of measured injection successes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。