针对大模型工具调用中的间接提示注入,构建动态红队测试基准并提出高效防御方案。
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations

- 设计215种跨24个企业集成的动态红队场景,覆盖五类攻击
- 无防护时攻击成功率高达32%~81%,新模型防御效果提升75-77个百分点
- 可迁移性强,适用于真实生产环境中的多集成、多攻击者场景
工具使用型大模型代理面临间接提示注入的实际威胁:代理通过工具调用读取第三方服务(如Gmail、Salesforce、Jira)的响应内容,而这些内容由用户编写或控制。现有基准测试严重低估该风险:多数仅覆盖少量集成,且重复使用相同攻击载荷;开源防护模型也仅在对话数据上训练,而非工具响应内容。本文提出AGENTREDBENCH,一个包含215个微妙未明确授权场景的动态红队测试基准,覆盖24个企业级集成和五类攻击类型。在八款主流模型(Anthropic、OpenAI、Google)上,无防护攻击成功率在32%至81%之间。为确保场景集不被训练数据污染并维持评估结果长期有效性,研究开源代码库、集成模式与AGENTREDGUARD模型;核心场景通过维护者中继通道进行不可变版本化评估。AGENTREDGUARD在三个目标模型族(Haiku、GPT-5.4-mini、Gemini-3-flash)上实现75-77个百分点的在线攻击成功率降低,真实良性误报率为0.0%(合成良性数据下为0.2%),优于所有开源基线(Llama Guard、PromptGuard 2、ProtectAI)在检测率与误报率上的表现。跨集成与跨攻击者留出验证表明性能优势具有泛化能力。
原文摘要 · Abstract (English)
Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither writes nor controls. Existing benchmarks under-measure the threat: most cover only a handful of integrations with the same attack payload replayed across runs, and open-source guards are trained on chat-style data rather than tool-response content. We introduce AGENTREDBENCH, a dynamic LLM-driven redteaming benchmark of 215 subtle underspecified-authorization scenarios across 24 enterprise integrations and five attack types. Across an eight-model panel (Anthropic, OpenAI, Google), no-guard attack success rate ranges from 32% to 81%. To keep the scenario set out of training corpora and preserve headline ASR meaning over time, we release the codebase, integration schemas, and AGENTREDGUARD model openly; the canonical scenarios are evaluated through a maintainer-mediated channel with immutable versioning. AGENTREDGUARD cuts online attack success by 75-77pp across three target model families (Haiku, GPT-5.4-mini, Gemini-3-flash) at 0.0% real-benign false-positive rate (0.2% on a synthetic-benign corpus), outperforming every open-source baseline with non-trivial detection (Llama Guard, PromptGuard 2, ProtectAI) on both axes. Cross-integration and cross-attacker holdouts (two independent attacker families held out from training) confirm the gain transfers beyond the training subset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。