arXiv:2601.12762cs.SEcs.AI2026-01被引 1

让大模型通过环境交互自主学习用工具,提升应对新工具的能力。

Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction

  • 用试错+执行的训练方式,让模型从模仿转向主动探索。
  • 在未见过的工具上表现显著优于现有方法,泛化能力更强。
  • 适合需要灵活使用外部工具的智能体开发场景。

赋予大语言模型外部工具可解决复杂现实问题,但现有方法在面对新或变化的工具时鲁棒性不足。当前以轨迹为中心的方法主要依赖训练中记忆固定解决方案路径,限制了模型对新工具的泛化能力。本文提出ToolMaster框架,将工具使用从模仿最优调用轨迹转变为通过与环境交互主动学习。该框架采用试错-执行范式:先让模型模仿包含显式试错与自我修正的教师轨迹,再通过强化学习联合优化试错与执行阶段。此过程使智能体能通过主动环境交互自主探索正确工具用法,并积累经验知识,从而提升工具执行效果。实验表明,ToolMaster在未见或陌生工具上的泛化性和鲁棒性显著优于现有基线。所有代码与数据已公开于https://github.com/NEUIR/ToolMaster。

原文摘要 · Abstract (English)

Equipping Large Language Models (LLMs) with external tools enables them to solve complex real-world problems. However, the robustness of existing methods remains a critical challenge when confronting novel or evolving tools. Existing trajectory-centric paradigms primarily rely on memorizing static solution paths during training, which limits the ability of LLMs to generalize tool usage to newly introduced or previously unseen tools. In this paper, we propose ToolMaster, a framework that shifts tool use from imitating golden tool-calling trajectories to actively learning tool usage through interaction with the environment. To optimize LLMs for tool planning and invocation, ToolMaster adopts a trial-and-execution paradigm, which trains LLMs to first imitate teacher-generated trajectories containing explicit tool trials and self-correction, followed by reinforcement learning to coordinate the trial and execution phases jointly. This process enables agents to autonomously explore correct tool usage by actively interacting with environments and forming experiential knowledge that benefits tool execution. Experimental results demonstrate that ToolMaster significantly outperforms existing baselines in terms of generalization and robustness across unseen or unfamiliar tools. All code and data are available at https://github.com/NEUIR/ToolMaster.

大模型工具使用强化学习泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。