用压缩嵌入替代完整工具列表,大幅降低上下文占用和延迟
NTILC: Neural Tool Invocation via Learned Compression

- 将工具与意图映射到共享嵌入空间,通过外部检索选工具
- 相比基线减少95%上下文消耗,推理延迟降低74%
- 适合大工具库场景,提升长上下文任务效率
依赖大量可调用API、函数和本地操作的智能体工具调用语言模型,将完整工具规格直接放入提示中会随工具库规模线性增加上下文开销,导致延迟上升且选择准确率下降,尤其受无关工具干扰。本文提出NTILC,一种基于学习的隐式检索工具选择与调用框架,以隐向量检索替代上下文内查找。该框架将用户意图与工具规范共同映射至共享嵌入空间,仅需被选工具的模式作为语言模型输入,实现精准参数生成。核心是签名感知复合目标,结合语义相似性与工具签名约束(如参数结构、类型兼容性、返回类型)。通过融合Circle Loss与功能边界损失,强化语义相近但执行签名不兼容工具间的区分。在公开工具选择与函数调用数据集上评估,相较长上下文基线,NTILC在所有设置中上下文令牌使用减少超95%,推理延迟降低最高达74%。
原文摘要 · Abstract (English)
Agentic tool-calling language models depend on large registries of callable APIs, functions, and local actions. Placing full tool specifications directly in the prompt incurs a cost that scales linearly with the size of the tool registry, rapidly consuming the context budget. As the registry grows, this leads to higher latency and degrades selection accuracy, particularly due to interference from irrelevant tools. We overcome these limitations by introducing NTILC, a neural tool selection and invocation framework that replaces in-context registry look-up with learned latent retrieval. NTILC maps both user intent and tool specifications into a shared embedding space, enabling tool selection via external retrieval rather than in-context lookup. The language model is conditioned only on the selected tool schema, allowing for precise, constrained argument generation. Central to our approach is a signature-aware composite objective, which augments semantic similarity with constraints derived from tool signatures (e.g., argument schema, type compatibility, and return types). By combining Circle Loss with a Functional Margin Loss, the model enforces separation between tools that are semantically similar but incompatible under their execution signatures. We evaluate NTILC on public tool-selection and function-calling datasets and report context token usage, retrieval accuracy, and selection latency metrics. Across these settings, NTILC reduces context window consumption by over 95% and inference latency by up to 74% compared to long-context ICT baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。