arXiv:2608.30362cs.AIcs.CL2026-08

提出新攻击方法,让大模型代理悄悄执行恶意指令而不留痕迹。

Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents

论文配图:Will the User Ever Know? Covert Indirect Prompt Injection Attacks on Tool-Using LLM Agents
图 1 · 摘自论文原文
  • 设计ICoA攻击,诱导代理在执行注入后返回用户任务,隐藏攻击痕迹。
  • 在AgentDojo上测试,相比最强基线,隐蔽成功率提升3.79至12.01个百分点。
  • 揭示了攻击隐蔽性与代理行为模式的关系,适合安全研究者参考。

随着大模型代理通过工具执行真实世界操作,间接提示注入(IPI)已成为严重威胁。标准指标攻击成功率(ASR)仅统计是否成功,却忽略用户能否在最终响应中察觉。分析成功注入轨迹发现两种结果:代理执行攻击但返回正常响应(隐蔽成功),或在最终响应中报告注入动作(明显成功)。我们据此将ASR分解为隐蔽成功率(CSR)和明显成功率(OSR)。分析表明,隐蔽与明显的区别源于注入后代理的行为:隐蔽路径在结束前将控制权交还给用户任务,而明显路径以攻击本身收尾。这一差异源于ReAct格式中最终响应总结最近动作的机制。基于此,我们提出ICoA(诱导隐蔽攻击),一种通过注入后引导代理返回用户任务来制造隐蔽结果的IPI攻击。在AgentDojo上的四类目标模型测试中,ICoA达到最高CSR,相比最强基线提升3.79至12.01个百分点。

原文摘要 · Abstract (English)

As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent's final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user's perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent's behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79-12.01 percentage points over the strongest baseline.

提示注入模型安全隐蔽攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。