让大模型自主决定何时用工具，提升推理能力。
AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning
- 用强化学习让模型自适应选择工具，不再依赖固定流程。
- 在多类任务中表现优于基线，工具使用更准确高效。
- 适合需要灵活调用工具的复杂推理场景研究者参考。
大型语言模型（LLMs）经由面向推理的后训练，演变为强大的大型推理模型（LRMs）。工具集成推理（TIR）通过引入外部工具进一步拓展其能力，但现有方法常依赖僵化的预设工具使用模式，可能损害核心语言能力。受人类动态选择工具能力启发，我们提出AutoTIR，一种基于强化学习的框架，使LLMs能在推理过程中自主决定是否以及使用哪个工具，而非遵循静态策略。AutoTIR采用混合奖励机制，联合优化任务答案正确性、结构化输出一致性及错误工具使用的惩罚，从而促进精确推理与高效工具融合。在多样化知识密集型、数学及通用语言建模任务上的广泛评估表明，AutoTIR实现显著优于基线的整体性能，并展现出更优的工具使用泛化能力。这些结果凸显强化学习在构建真正可泛化、可扩展的TIR能力中的潜力。代码与数据已公开于https://github.com/weiyifan1023/AutoTIR。
原文摘要 · Abstract (English)
Large Language Models (LLMs), when enhanced through reasoning-oriented post-training, evolve into powerful Large Reasoning Models (LRMs). Tool-Integrated Reasoning (TIR) further extends their capabilities by incorporating external tools, but existing methods often rely on rigid, predefined tool-use patterns that risk degrading core language competence. Inspired by the human ability to adaptively select tools, we introduce AutoTIR, a reinforcement learning framework that enables LLMs to autonomously decide whether and which tool to invoke during the reasoning process, rather than following static tool-use strategies. AutoTIR leverages a hybrid reward mechanism that jointly optimizes for task-specific answer correctness, structured output adherence, and penalization of incorrect tool usage, thereby encouraging both precise reasoning and efficient tool integration. Extensive evaluations across diverse knowledge-intensive, mathematical, and general language modeling tasks demonstrate that AutoTIR achieves superior overall performance, significantly outperforming baselines and exhibits superior generalization in tool-use behavior. These results highlight the promise of reinforcement learning in building truly generalizable and scalable TIR capabilities in LLMs. The code and data are available at https://github.com/weiyifan1023/AutoTIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。