首个评测大模型工具调用安全性的基准，揭示攻击漏洞。
MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents
- 构建12类攻击分类，覆盖工具命名冲突、提示注入等
- 实测9个主流模型在405个工具上，2000次攻击实例
- 提出安全-性能权衡指标，适合安全与应用研究者
模型上下文协议（MCP）标准化了大语言模型（LLM）代理发现、描述和调用外部工具的方式。尽管提升了互操作性，但也因将工具作为可组合对象并暴露自然语言元数据和标准化输入输出而扩大了攻击面。本文提出MSB（MCP安全基准），首个端到端评估套件，系统衡量LLM代理在完整工具使用流程中对MCP特有攻击的抵御能力：任务规划、工具调用和响应处理。贡献包括：(1) 12类攻击分类，涵盖名称冲突、偏好操控、嵌入工具描述中的提示注入、越界参数请求、用户伪装响应、错误升级、工具转移、检索注入及混合攻击；(2) 实际执行攻击的评估框架，通过MCP运行真实工具（良性与恶意）而非模拟；(3) 安全性与性能权衡度量指标——净韧性性能（NRP）。在10个领域、405个工具上评估9个主流LLM代理，生成2000个攻击实例。结果表明攻击在每个环节均有效，性能越强的模型越易受攻击，因其工具调用与指令遵循能力突出。MSB为研究人员和实践者提供了实用基准，用于研究、比较和强化MCP代理。代码：https://github.com/dongsenzhang/MSB
原文摘要 · Abstract (English)
The Model Context Protocol (MCP) standardizes how large language model (LLM) agents discover, describe, and call external tools. While MCP unlocks broad interoperability, it also enlarges the attack surface by making tools first-class, composable objects with natural-language metadata, and standardized I/O. We present MSB (MCP Security Benchmark), the first end-to-end evaluation suite that systematically measures how well LLM agents resist MCP-specific attacks throughout the full tool-use pipeline: task planning, tool invocation, and response handling. MSB contributes: (1) a taxonomy of 12 attacks including name-collision, preference manipulation, prompt injections embedded in tool descriptions, out-of-scope parameter requests, user-impersonating responses, false-error escalation, tool-transfer, retrieval injection, and mixed attacks; (2) an evaluation harness that executes attacks by running real tools (both benign and malicious) via MCP rather than simulation; and (3) a robustness metric that quantifies the trade-off between security and performance: Net Resilient Performance (NRP). We evaluate nine popular LLM agents across 10 domains and 405 tools, producing 2,000 attack instances. Results reveal the effectiveness of attacks against each stage of MCP. Models with stronger performance are more vulnerable to attacks due to their outstanding tool calling and instruction following capabilities. MSB provides a practical baseline for researchers and practitioners to study, compare, and harden MCP agents. Code: https://github.com/dongsenzhang/MSB
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。