让大模型内化工具知识，提升推理效率与准确性
TInR: Exploring Tool-Internalized Reasoning in Large Language Models
- 通过双向对齐将工具知识融入模型内部
- 三阶段训练使模型在内外部场景均表现更优
- 适合需要高效精准推理的AI应用开发者
工具集成推理（TIR）通过在推理过程中调用外部工具扩展大语言模型的能力。现有方法依赖外部工具文档，导致工具掌握困难、规模受限且推理效率低。为此，本文提出工具内化推理（TInR），旨在将工具知识内置于大模型中。为实现该目标，设计了三阶段训练框架TInR-U：1）采用双向知识对齐策略完成工具内化；2）使用高质量推理标注进行监督微调预热；3）引入TInR特有奖励机制进行强化学习。在域内与域外设置下全面评估，结果表明TInR-U在两类场景中均取得更优性能，验证了其有效性和高效性。
原文摘要 · Abstract (English)
Tool-Integrated Reasoning (TIR) has emerged as a promising direction by extending Large Language Models' (LLMs) capabilities with external tools during reasoning. Existing TIR methods typically rely on external tool documentation during reasoning. However, this leads to tool mastery difficulty, tool size constraints, and inference inefficiency. To mitigate these issues, we explore Tool-Internalized Reasoning (TInR), aiming at facilitating reasoning with tool knowledge internalized into LLMs. Achieving this goal presents notable requirements, including tool internalization and tool-reasoning coordination. To address them, we propose TInR-U, a tool-internalized reasoning framework for unified reasoning and tool usage. TInR-U is trained through a three-phase pipeline: 1) tool internalization with a bidirectional knowledge alignment strategy; 2) supervised fine-tuning warm-up using high-quality reasoning annotations, and 3) reinforcement learning with TInR-specific rewards. We comprehensively evaluate our method across in-domain and out-of-domain settings. Experiment results show that TInR-U achieves superior performance in both settings, highlighting its effectiveness and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。