arXiv:2505.01441cs.AI2025-05被引 98

让大模型学会自主调用工具并决策,通过强化学习提升复杂问题解决能力。

Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning

  • 通过强化学习让模型自主决定何时、如何使用外部工具
  • 在数学推理和函数调用任务上比基线模型提升22%准确率
  • 适合需要多步推理与工具协同的AI系统研发人员

大型语言模型在复杂推理任务中取得了显著进展,但仍受限于静态内部知识和仅文本推理。真实世界的问题求解往往需要动态、多步推理、自适应决策以及与外部工具和环境交互的能力。本文提出ARTIST(自进化变压器中的代理式推理与工具集成),一个统一框架,将代理式推理、强化学习与工具集成紧密结合。ARTIST使模型能够在多轮推理链中自主决定何时、如何及使用何种工具,利用基于结果的强化学习,在无需逐步监督的情况下学习稳健的工具使用策略。在数学推理和多轮函数调用基准上的大量实验表明,ARTIST持续优于当前最优基线,相较于基础模型最高提升22%绝对性能,并在最挑战性任务上取得显著增益。详细研究与指标分析显示,代理式强化学习训练带来了更深层次的推理、更高效的工具使用和更高质量的解决方案。结果确立了带有工具集成的代理式强化学习作为大模型鲁棒、可解释且泛化性强问题求解的新前沿。

原文摘要 · Abstract (English)

Large language models (LLMs) have achieved remarkable progress in complex reasoning tasks, yet they remain fundamentally limited by their reliance on static internal knowledge and text-only reasoning. Real-world problem solving often demands dynamic, multi-step reasoning, adaptive decision making, and the ability to interact with external tools and environments. In this work, we introduce ARTIST (Agentic Reasoning and Tool Integration in Self-improving Transformers), a unified framework that tightly couples agentic reasoning, reinforcement learning, and tool integration for LLMs. ARTIST enables models to autonomously decide when, how, and which tools to invoke within multi-turn reasoning chains, leveraging outcome-based RL to learn robust strategies for tool use and environment interaction without requiring step-level supervision. Extensive experiments on mathematical reasoning and multi-turn function calling benchmarks show that ARTIST consistently outperforms state-of-the-art baselines, with up to 22% absolute improvement over base models and strong gains on the most challenging tasks. Detailed studies and metric analyses reveal that agentic RL training leads to deeper reasoning, more effective tool use, and higher-quality solutions. Our results establish agentic RL with tool integration as a powerful new frontier for robust, interpretable, and generalizable problem-solving in LLMs.

强化学习工具集成推理能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。