研究大模型工具调用的有效性与训练效率，发现评测细节影响性能排名，提出加速训练新方法。
On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
- 分析工具调用评测流程，指出随机种子、提示模板等细节显著影响结果
- 发现强化学习训练中存在大量无效推理和高计算开销，导致效率低下
- 提出两项优化技术，实现加速训练且不降低性能，适合追求高效训练的开发者
工具调用是现代大语言模型代理的核心能力，使其具备超越参数知识的技能。本文从有效性（如何衡量）与效率（如何学习）两个互补维度研究工具调用。在有效性方面，系统分析了工具调用评估流程，发现结果对看似微小但常未文档化的实现选择（如随机种子、系统提示、多轮模板构建、历史信息传递方式）高度敏感，这些因素在多轮场景下可导致性能报告差异显著，缺乏严格标准化时排行榜不可靠。在效率方面，考察标准强化学习（RL）用于工具调用时，发现两处计算浪费：(i) 推理过程中许多提示无学习信号；(ii) 策略更新时优化成本高。基于此，提出两种技术加速基于RL的工具调用训练，在不降性能的前提下实现显著的墙钟时间提速。
原文摘要 · Abstract (English)
Tool-calling is a central component of modern large language model (LLM) agents, equipping them with skills beyond their parametric knowledge. This paper studies tool-calling along two complementary axes: effectiveness, i.e., how this capability is measured, and efficiency, i.e., how it is learned. On effectiveness, we systematically analyze tool-calling evaluation pipelines and show that results can be highly sensitive to seemingly minor, often undocumented implementation choices including the random seed, system prompt, multi-turn template construction, and how prior interaction/reasoning history is carried forward. These choices can lead to substantial differences in reported performance, especially in multi-turn settings where without rigorous standardization, leaderboard rankings are unreliable. On efficiency, we examine standard reinforcement learning (RL) for tool-calling and identify two sources of computational waste: (i) during rollouts, many prompts produce no learning signal, and (ii) during policy updates, optimization incurs high computational cost. Guided by these findings, we introduce two techniques that accelerate RL-based tool-calling training, achieving substantial wall-clock speedup without degrading performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。