用分类体系精简大模型提示词,降低计算成本并提升工具选择准确率。
JSPLIT: A Taxonomy-based Solution for Prompt Bloating in Model Context Protocol
- 构建工具分类树,根据用户请求筛选相关工具,避免冗余信息。
- 在100+工具场景下,提示词长度减少62%,任务成功率提升18%。
- 适合需要调用多种外部工具的复杂智能体系统开发者使用。
AI系统持续演进,用户需求已超越简单的文本交互,转向需与外部工具协作的复杂智能体系统。为支持此类应用,模型上下文协议(MCP)应运而生,通过在提示中包含工具能力描述实现工具调用。然而,随着工具数量增加,提示词不断膨胀,导致高令牌成本、延迟上升及因选择无关工具而降低任务成功率。为此,我们提出JSPLIT——一种基于分类体系的框架,将工具组织为层级结构,结合用户提示与分类信息,仅引入最相关的工具。本文阐述了分类设计、工具选择算法及评估数据集。实验表明,JSPLIT显著缩减提示长度,且在工具数量增至100+时,反而提升工具选择准确率,有效降低开销并提高复杂环境下的任务成功率。
原文摘要 · Abstract (English)
AI systems are continually evolving and advancing, and user expectations are concurrently increasing, with a growing demand for interactions that go beyond simple text-based interaction with Large Language Models (LLMs). Today's applications often require LLMs to interact with external tools, marking a shift toward more complex agentic systems. To support this, standards such as the Model Context Protocol (MCP) have emerged, enabling agents to access tools by including a specification of the capabilities of each tool within the prompt. Although this approach expands what agents can do, it also introduces a growing problem: prompt bloating. As the number of tools increases, the prompts become longer, leading to high prompt token costs, increased latency, and reduced task success resulting from the selection of tools irrelevant to the prompt. To address this issue, we introduce JSPLIT, a taxonomy-driven framework designed to help agents manage prompt size more effectively when using large sets of MCP tools. JSPLIT organizes the tools into a hierarchical taxonomy and uses the user's prompt to identify and include only the most relevant tools, based on both the query and the taxonomy structure. In this paper, we describe the design of the taxonomy, the tool selection algorithm, and the dataset used to evaluate JSPLIT. Our results show that JSPLIT significantly reduces prompt size without significantly compromising the agent's ability to respond effectively. As the number of available tools for the agent grows substantially, JSPLIT even improves the tool selection accuracy of the agent, effectively reducing costs while simultaneously improving task success in high-complexity agent environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。