解决大模型工具调用参数错误问题,提升真实场景下工具使用成功率。
Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls

- 用隐藏状态中的正确性信号指导参数生成
- 参数准确率从19.7%提升至59.6%
- 适合需要高精度工具调用的系统开发人员
大型语言模型代理的性能很大程度上依赖于工具调用。现有研究多关注工具选择与调用顺序,但参数填写的准确性同样关键,却长期被忽视。在云网络等复杂领域,前沿模型正确完成的工具调用不足一半。受近期研究启发,我们发现模型生成参数时,其隐藏状态中蕴含强正确性信号:简单线性探测器可准确预测值是否正确。基于此,提出统一的探针引导框架,包含两种互补方法:探针过滤自举训练(PBT),利用探针筛选可靠自生成调用用于微调;探针引导重排序(PGR),在推理阶段用探针优选更优候选。为支持系统评估,发布ParamBench,基于真实云网络API构建,按参数嵌套深度、跨参数依赖及推理需求分为五个难度等级。在5个开源模型上对ParamBench及6个外部基准的实验表明,该方法显著提升参数生成能力,平均精确匹配率从19.7%提升至59.6%。
原文摘要 · Abstract (English)
Large language model agents derive much of their capability from tool use. Existing research on tool use has largely focused on selecting the right tool and orchestrating the order of calls. However, correctly filling the parameters of a tool call is equally critical for successful execution and has received far less attention. In domains such as cloud networking, even frontier models correctly complete fewer than half of tool calls. Inspired by recent analyses showing that LLM hidden states encode rich information about model predictions, we discover that while the model generates a parameter value, its hidden state contains a strong correctness signal: a simple linear probe can accurately predict whether the value will be correct. Based on this observation, we propose a unified probe-guided framework with two complementary approaches: probe-filtered bootstrapped training (PBT), which uses the probe to filter reliable self-generated calls for fine-tuning, and probe-guided reranking (PGR), which uses the probe to select better candidates during inference. To support systematic evaluation, we release ParamBench, a benchmark built from real cloud-network APIs that categorizes every instance into five difficulty levels according to parameter nesting depth, cross-parameter dependencies, and the reasoning required to derive values from earlier calls. Extensive experiments across 5 open models on ParamBench and 6 external benchmarks demonstrate that our method substantially improves parameter generation, raising the average exact match from 19.7% to 59.6%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。