测试大模型是否用服务器嵌入的数据，发现多数能用但易被搜索功能干扰。
LLM within MCP Matters: Measuring Inefficient Resource Utilization Driven by LLMs
- 在指令中预置数据，让大模型直接读取而非搜索。
- 无搜索时23/24模型读取成功率超98%，有搜索时9个模型降至15%以下。
- 需优化提示工程顺序，建议系统明确优先使用嵌入数据。
模型上下文协议（MCP）标准化了服务器向大语言模型（LLMs）暴露数据和工具的方式。常见设计将常用参考数据（如标识符查表）直接嵌入服务器指令中，即服务器提供给客户端应用的系统提示文本。当查询涉及嵌入表中的条目时，模型可立即响应，无需通过搜索工具重新发现信息。我们对24个大模型（9个Claude、6个Gemini、9个GPT）在生产级法律信息MCP服务器上进行了54,000次试验，检验客户端大模型是否实际使用这些嵌入数据。诊断实验显示：移除竞争性搜索工具后，失败主要源于行为偏好而非能力缺失。在无搜索工具情况下，23/24个模型可靠读取嵌入数据（命中率≥98%）；而仅存在搜索工具时，9个模型命中率低于15%。三因素（2³）析因分析表明存在显著交互效应：三种指令干预组合可使20/24个模型恢复至少86%的命中率，但单独使用可能对特定模型家族产生反效果。因此，针对每个服务器的提示工程只是权宜之计；我们主张MCP宿主应用应提供显式机制，确保服务器指令在客户端大模型决策中优先于工具选择。
原文摘要 · Abstract (English)
The Model Context Protocol (MCP) standardizes how servers expose data and tools to Large Language Models (LLMs). A common server design embeds frequently used reference data, such as identifier lookup tables, directly in the server instructions: the system-prompt text a server hands to the host application. When a query concerns an entry of the embedded table, the model can act on it immediately instead of re-discovering the same information through a search tool. We test whether client LLMs actually consume such instruction-embedded data, reporting a 54,000-trial study across 24 LLMs (9 Claude, 6 Gemini, 9 GPT) on a production legal-information MCP server. A diagnostic condition that removes the competing search tool shows that failures are dominated by behavioral preference rather than missing capability. With search unavailable, 23 of 24 models read the embedded data reliably (hit ratio at least 98%); with a search tool merely present, 9 models drop below 15%. A 2^3 factorial analysis of three instruction-level interventions reveals strong interaction effects: combining all three restores at least 86% for 20 of 24 models, but individual interventions can backfire for specific model families. Per-server prompt engineering is therefore a workaround rather than a fix; we argue that MCP host applications should provide an explicit mechanism that places server instructions ahead of tool selection in the client LLM's deliberation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。