arXiv:2505.05849cs.CRcs.AI2025-05被引 43

自动发现大模型代理的间接提示注入漏洞,提升攻击成功率近一倍。

AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents

  • 基于MCTS的种子筛选与迭代优化,实现黑盒模糊测试。
  • 在两个基准上对o3-mini和GPT-4o的成功率分别达71%和70%。
  • 可跨任务、跨模型迁移,适用于真实场景中的安全测试。

大型语言模型(LLM)强大的规划与推理能力推动了基于代理系统的兴起,使其能够调用外部工具并交互于复杂环境。然而,这些功能也带来了关键安全风险:间接提示注入,即通过操纵上下文信息而非直接用户输入来攻击代理核心的LLM。本文提出通用黑盒模糊测试框架AgentVigil,可自动发现并利用多种LLM代理中的此类漏洞。该方法首先构建高质量初始种子语料库,再采用基于蒙特卡洛树搜索(MCTS)的种子选择算法,迭代优化输入以最大化漏洞暴露概率。在AgentDojo和VWA-adv两个公开基准上,AgentVigil对o3-mini和GPT-4o驱动的代理分别达到71%和70%的成功率,近乎是基线攻击的两倍。此外,该框架在未见任务和内部模型间表现出强迁移性,并对防御机制展现出良好效果。在真实环境中,其成功诱导代理访问任意网址,包括恶意站点。

原文摘要 · Abstract (English)

The strong planning and reasoning capabilities of Large Language Models (LLMs) have fostered the development of agent-based systems capable of leveraging external tools and interacting with increasingly complex environments. However, these powerful features also introduce a critical security risk: indirect prompt injection, a sophisticated attack vector that compromises the core of these agents, the LLM, by manipulating contextual information rather than direct user prompts. In this work, we propose a generic black-box fuzzing framework, AgentVigil, designed to automatically discover and exploit indirect prompt injection vulnerabilities across diverse LLM agents. Our approach starts by constructing a high-quality initial seed corpus, then employs a seed selection algorithm based on Monte Carlo Tree Search (MCTS) to iteratively refine inputs, thereby maximizing the likelihood of uncovering agent weaknesses. We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o, respectively, nearly doubling the performance of baseline attacks. Moreover, AgentVigil exhibits strong transferability across unseen tasks and internal LLMs, as well as promising results against defenses. Beyond benchmark evaluations, we apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.

提示注入安全测试智能体黑盒攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。