通过动态工具调用反馈,发现大模型代理中隐蔽的注入攻击漏洞。
VeriGrey: Greybox Agent Validation
- 利用工具调用序列作为反馈驱动测试,捕捉罕见危险行为。
- 在AgentDojo上比黑盒方法多发现33%间接提示注入漏洞。
- 可有效检测真实代理中的恶意技能,适合安全评估与框架构建。
智能体人工智能近年来备受关注。大语言模型(LLM)代理在后端使用一个或多个LLM,前端通过结合LLM输出与外部工具调用结果实现自主决策。与外部环境的自主交互引入了关键安全风险。本文提出灰盒方法VeriGrey,通过工具调用序列作为反馈函数驱动测试过程,以发现罕见但危险的工具调用及意外行为。测试中,通过将任务与注入任务关联,设计具有危害性的注入提示,并作为完成代理功能的必要步骤。在包含GPT-4.1后端的AgentDojo基准上,相比黑盒基线,VeriGrey额外提升了33%的漏洞发现效率。在真实案例中,针对广泛使用的代码代理Gemini CLI和知名个人助手OpenClaw,VeriGrey识别出黑盒方法无法发现的攻击场景。在OpenClaw中,通过构造采用变异模糊测试的对话代理,成功从10个恶意技能中检测出10/10(100%)在Kimi-K2.5后端,9/10(90%)在Opus 4.6后端,验证了动态测试方法的价值,推动建立代理可信保障体系。
原文摘要 · Abstract (English)
Agentic AI has been a topic of great interest recently. A Large Language Model (LLM) agent involves one or more LLMs in the back-end. In the front end, it conducts autonomous decision-making by combining the LLM outputs with results obtained by invoking several external tools. The autonomous interactions with the external environment introduce critical security risks. In this paper, we present a grey-box approach to explore diverse behaviors and uncover security risks in LLM agents. Our approach VeriGrey uses the sequence of tools invoked as a feedback function to drive the testing process. This helps uncover infrequent but dangerous tool invocations that cause unexpected agent behavior. As mutation operators in the testing process, we mutate prompts to design pernicious injection prompts. This is carefully accomplished by linking the task of the agent to an injection task, so that the injection task becomes a necessary step of completing the agent functionality. Comparing our approach with a black-box baseline on the well-known AgentDojo benchmark, VeriGrey achieves 33% additional efficacy in finding indirect prompt injection vulnerabilities with a GPT-4.1 back-end. We also conduct real-world case studies with the widely used coding agent Gemini CLI, and the well-known OpenClaw personal assistant. VeriGrey finds prompts inducing several attack scenarios that could not be identified by black-box approaches. In OpenClaw, by constructing a conversation agent which employs mutational fuzz testing as needed, VeriGrey is able to discover malicious skill variants from 10 malicious skills (with 10/10= 100% success rate on the Kimi-K2.5 LLM backend, and 9/10= 90% success rate on Opus 4.6 LLM backend). This demonstrates the value of a dynamic approach like VeriGrey to test agents, and to eventually lead to an agent assurance framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。