用向量检索精简LLM工具调用,大幅降低消耗且保持高准确率。
Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection
- 通过语义向量索引工具,按用户意图动态选最相关的3-5个工具。
- 工具令牌消耗减少99.6%,3个候选下命中率达97.1%,平均响应<100ms。
- 适合需要高效调用大量外部工具的智能体与多组织系统场景。
具备工具调用能力的大语言模型(LLMs)在通过外部工具集成执行复杂任务方面展现出巨大潜力。模型上下文协议(MCP)已成为连接LLMs与多样化工具集的标准框架,单个MCP服务器可能提供数十至数百个工具。然而,当前实现面临关键可扩展性挑战:将所有可用工具提供给LLM上下文会导致显著的令牌开销、成本增加、准确率下降以及上下文窗口限制。本文提出一种基于语义的工具发现架构,通过向量检索解决上述问题。该方法使用密集嵌入对MCP工具进行索引,捕捉工具能力与用户意图间的语义关系,仅动态选择最相关的工具(通常为3-5个),而非暴露整个工具目录(50-100+个)。实验结果表明,在涵盖121个工具、来自5个MCP服务器的140个查询基准上,工具相关令牌消耗降低99.6%,在K=3时命中率达到97.1%,平均倒数排名(MRR)为0.91,检索延迟低于100毫秒。贡献包括:(1) MCP工具的语义索引框架,(2) 基于查询-工具相似性的动态工具选择算法,(3) 全面评估验证了显著的效率与准确性提升,(4) 可扩展至多智能体与跨组织工具发现。
原文摘要 · Abstract (English)
Large Language Models (LLMs) with tool-calling capabilities have demonstrated remarkable potential in executing complex tasks through external tool integration. The Model Context Protocol (MCP) has emerged as a standardized framework for connecting LLMs to diverse toolsets, with individual MCP servers potentially exposing dozens to hundreds of tools. However, current implementations face a critical scalability challenge: providing all available tools to the LLM context results in substantial token overhead, increased costs, reduced accuracy, and context window constraints. We present a semantic tool discovery architecture that addresses these challenges through vector-based retrieval. Our approach indexes MCP tools using dense embeddings that capture semantic relationships between tool capabilities and user intent, dynamically selecting only the most relevant tools (typically 3-5) rather than exposing the entire tool catalog (50-100+). Experimental results demonstrate a 99.6% reduction in tool-related token consumption with a hit rate of 97.1% at K=3 and an MRR of 0.91 on a benchmark of 140 queries across 121 tools from 5 MCP servers, with sub-100ms retrieval latency. Contributions include: (1) a semantic indexing framework for MCP tools, (2) a dynamic tool selection algorithm based on query-tool similarity, (3) comprehensive evaluation demonstrating significant efficiency and accuracy improvements, and (4) extensibility to multi-agent and cross-organizational tool discovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。