提升大模型函数调用的精准度，通过细粒度奖励模型优化每一步决策。
ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling
- 引入细粒度奖励模型ToolPRM，逐步评估函数名与参数选择
- 在多个基准上显著提升函数调用准确率，测试时稳定增益
- 揭示结构化生成需多探索但少保留，早期错误不可挽回
大型语言模型在函数调用任务中表现优异，但推理扩展主要集中在非结构化生成。本文提出一种面向结构化输出的推理扩展框架，结合细粒度束搜索与工具型过程奖励模型（ToolPRM），用于评分每个调用步骤中的决策（函数名与参数填充）。我们通过函数掩码、回滚收集和步骤级标注构建了首个细粒度调用监督数据集。ToolPRM在预测准确性上优于结果级与粗粒度奖励模型，并在多个函数调用基准上实现一致的测试时性能提升。进一步研究表明，结构化生成遵循‘探索更多但保留更少’的规律，因为早期的JSON错误无法恢复。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at function calling, but inference scaling has been explored mainly for unstructured generation. We propose an inference-scaling framework for structured outputs that combines fine-grained beam search with \textbf{ToolPRM}, a process reward model scoring each intra-call decision (function name and argument filling). We build the first fine-grained intra-call supervision dataset via function masking, rollout collection, and step-level annotation. ToolPRM outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. We further show that structured generation follows ``\textbf{explore more but retain less}'', since early JSON errors are unrecoverable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。