arXiv:2508.02110cs.AI2025-08NeurIPS被引 32

黑客通过伪装工具元数据,诱导大模型调用恶意工具。

Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools

  • 用迭代优化生成看似合法的诱人工具信息
  • 攻击成功率81%-95%,主任务几乎不受影响
  • 可绕过多种防御,适合研究安全的开发者看

大型语言模型(LLM)代理通过调用外部工具在复杂推理和决策中表现出色。然而,这种以工具为中心的范式引入了一个此前未被充分关注的攻击面:攻击者可通过操纵工具的元数据(如名称、描述、参数模式)来影响代理行为。我们将其识别为一种新型且隐蔽的威胁,使恶意工具能被优先选择,无需提示注入或访问模型内部。为此,我们提出吸引性元数据攻击(AMA),一个黑盒上下文学习框架,通过迭代优化生成高度吸引人但语法和语义上有效的工具元数据。该攻击可无缝集成至标准工具生态,无需修改代理执行框架。在十种真实模拟工具使用场景和多种主流LLM代理上的实验表明,攻击成功率高达81%–95%,并造成显著隐私泄露,对主任务执行影响极小。即使面对提示层防御、审计检测和结构化工具选择协议(如模型上下文协议),攻击依然有效,暴露出当前代理架构的系统性漏洞。这些发现表明,元数据操纵构成了一种强大且隐蔽的攻击面。值得注意的是,AMA与注入攻击正交,可结合使用以增强攻击效果,凸显了超越提示层和审计机制的执行层防御必要性。代码已开源:https://github.com/SEAIC-M/AMA。

原文摘要 · Abstract (English)

Large language model (LLM) agents have demonstrated remarkable capabilities in complex reasoning and decision-making by leveraging external tools. However, this tool-centric paradigm introduces a previously underexplored attack surface, where adversaries can manipulate tool metadata -- such as names, descriptions, and parameter schemas -- to influence agent behavior. We identify this as a new and stealthy threat surface that allows malicious tools to be preferentially selected by LLM agents, without requiring prompt injection or access to model internals. To demonstrate and exploit this vulnerability, we propose the Attractive Metadata Attack (AMA), a black-box in-context learning framework that generates highly attractive but syntactically and semantically valid tool metadata through iterative optimization. The proposed attack integrates seamlessly into standard tool ecosystems and requires no modification to the agent's execution framework. Extensive experiments across ten realistic, simulated tool-use scenarios and a range of popular LLM agents demonstrate consistently high attack success rates (81\%-95\%) and significant privacy leakage, with negligible impact on primary task execution. Moreover, the attack remains effective even against prompt-level defenses, auditor-based detection, and structured tool-selection protocols such as the Model Context Protocol, revealing systemic vulnerabilities in current agent architectures. These findings reveal that metadata manipulation constitutes a potent and stealthy attack surface. Notably, AMA is orthogonal to injection attacks and can be combined with them to achieve stronger attack efficacy, highlighting the need for execution-level defenses beyond prompt-level and auditor-based mechanisms. Code is available at https://github.com/SEAIC-M/AMA.

大模型安全工具调用攻击防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。