arXiv:2605.24069cs.CRcs.AI2026-05被引 1

攻击者通过篡改工具说明书,让大模型代理误执行恶意操作。

When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents

论文配图:When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents
图 1 · 摘自论文原文
  • 攻击者在工具描述文档中埋入恶意指令,而非代码,隐蔽性强。
  • 8个主流大模型在6类高危场景中攻击成功率接近100%。
  • 提出自我修正机制,可自动识别并回滚错误行为,适合安全研究者。

随着以模型上下文协议(MCP)为代表的标准化协议推动,具备工具调用能力的大语言模型(LLM)代理实现了前所未有的自主执行能力,其通过集成外部开放域知识与工具实现高效规划。然而,这种互操作性也带来了针对代理认知规划层的隐蔽攻击面。本文系统研究了一种新型语义攻击——工具描述中毒(TDP):恶意指令不嵌入工具可执行代码,而是隐匿于代理依赖的描述性元数据(即“说明书”)中。为严谨评估该威胁,我们构建了首个专用于TDP的基准测试平台MCP-TDP Security Benchmark,包含32个真实世界测试案例,覆盖6类风险场景。对8个主流大模型的评估显示,领先模型如GPT-4o在6个高风险场景中攻击成功率(ASR)接近100%。此外,常见提示防护机制几乎无效,甚至可能适得其反(我们称之为“防火墙悖论”)。关键的是,我们提出一种防御机制——“反应式自我修正”,使代理可在执行后自主检测并回滚恶意行为。本工作首次提供面向TDP的专用安全基准,为保障先进代理系统的认知与规划层安全提供关键洞见。

原文摘要 · Abstract (English)

The rise of tool-using Large Language Model (LLM) agents, standardized by protocols like the Model Context Protocol (MCP), has unlocked unprecedented autonomous execution capabilities for LLM Agents by integrating external open-domain knowledge and tools. However, this interoperability introduces a covert attack surface targeting the agent's cognitive planning layer. This paper systematically investigates Tool Description Poisoning (TDP), a novel semantic attack. In TDP, malicious instructions are not embedded in a tool's executable code, but rather covertly injected into its descriptive metadata, the very "manual" an agent relies on for secure planning and decision-making. To rigorously and systematically evaluate this emerging threat, we introduce the MCP-TDP Security Benchmark. This high-fidelity sandbox environment comprises 32 realistic, real-world test cases spanning 6 distinct risk categories. Our evaluation of 8 mainstream LLMs reveals severe vulnerabilities, with leading models like GPT-4o exhibiting a nearly 100% Attack Success Rate (ASR) in six high-risk scenarios. Furthermore, our findings demonstrate that common prompt-guardrail defenses are largely ineffective and can, counterintuitively, even be counterproductive (a phenomenon which we term the "Firewall Fallacy"). Crucially, we also propose a defense mechanism: "Reactive Self-Correction," where an agent autonomously detects and reverts its own malicious actions post-execution. This work provides the first specialized security benchmark tailored for TDP, offering essential insights for securing the cognitive and planning layers of advanced agentic systems.

安全评测大模型代理语义攻击自我修正

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。