让企业大模型高效精准调用8000+工具,且推理速度够快可直接上线。
TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs

- 分两阶段训练:先注入工具知识,再学推理链条
- 单步贪婪解码下召回率超86%,比基线提升近60个百分点
- 适合需要快速部署、高精度工具调用的企业级LLM系统
参数化检索通过为每个API分配唯一虚拟标记,让大模型在约束束搜索下生成该标记来隐式调用工具。但现有方法存在两大缺陷:训练中会破坏工具知识,且束搜索解码速度过慢难以实时部署。本文提出TRACE(基于业务规则的推理课程),采用两阶段课程训练解决此问题。第一阶段复用ToolSense的多格式记忆微调数据,结合LoRA注入工具知识。第二阶段为核心创新:模型需在输出工具标记列表前生成思维链,使用两类数据——ToolSense的问答对和领域专家定制的业务规则合成查询,并均附加推理轨迹。该训练在保持第一阶段的多选题与问答探针准确率基础上,实现生产级单步贪婪解码。在包含8300+工具的两个企业产品线联合数据集上评估,第二阶段训练不仅保留更优工具理解:多选题准确率提升3.2个百分点,问答探针提升9个百分点;在检索任务中,域A召回率达约86%,域B达约60%,远超嵌入基线的约27%和约52%,且均采用单步贪婪解码,可直接投入生产。
原文摘要 · Abstract (English)
Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search. Toolsense shows that this regime has two critical drawbacks: it destroys parametric tool knowledge during training, and its beam-search decoding is too slow for real-time deployment. We introduce TRACE (Tool Retrieval via Augmented Chain-of-thought and Enterprise rules), a two-stage curriculum that resolves this dissociation. Stage 1 reuses the multi-format memorization SFT from ToolSense to seed tool knowledge with LoRA. Stage 2 is our core contribution: the model is trained to emit a thinking trace before producing a JSON list of tool tokens, using two data sources -- RRB pairs from ToolSense and queries synthesized to target business rules curated by domain experts -- both augmented with reasoning traces. This training objective preserves Stage 1 MCQ and QA probing accuracy while enabling single-beam greedy decoding at production latency. Evaluated on a combined enterprise catalog of 8,300+ tools across two enterprise product lines, TRACE training for Stage 2 not only preserves but improves tool understanding: MCQ accuracy gains +3.2 pp and QA probing gains +9 pp over Stage 1. On retrieval, TRACE achieves ~86% recall on Domain A and ~60% on Domain B -- compared to embedding baseline performance of ~27% & ~52% -- both with single-beam greedy decoding, making it directly deployable at production latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。