让大模型学会判断何时用工具，减少97.6%无效调用
AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization
- 根据任务难易动态调整工具使用，简单任务内部推理
- 工具调用减少97.6%（简单任务），准确率不降反升
- 适合追求高效推理、减少外部依赖的研究者
工具集成推理（TIR）显著提升了大语言模型的能力，但现有智能体常出现认知卸载问题，即使面对简单任务也冗余调用外部工具。本文提出AdaTIR框架，将静态工具调用转变为基于难度感知的推理内化。通过引入难度感知效率奖励，AdaTIR根据任务复杂度动态调整工具预算——简单任务内部推理，复杂任务选择性调用工具。此外，我们识别出工具惩罚超过正确性奖励的符号反转问题，导致正确推理路径被错误惩罚。为此提出截断优势塑造（CAS），确保正确性为首要目标，效率为次要约束。实验表明，AdaTIR在简单任务上减少97.6%工具调用，在复杂任务上减少28.2%，同时保持或提升准确率。尤为关键的是，即使禁用工具访问，AdaTIR在AIME 2024上仍比基线高4.8%，成功实现推理内化。
原文摘要 · Abstract (English)
Tool-Integrated Reasoning (TIR) has significantly enhanced the capabilities of Large Language Models (LLMs), yet current agents tend to exhibit cognitive offloading, redundantly invoking external tools even for simple tasks. In this paper, we suggest that true agentic intelligence requires not just tool invocation, but the adaptive wisdom to discern when to use them. We propose AdaTIR, a framework that shifts the paradigm from static tool invocation to difficulty-aware reasoning internalization. By introducing a difficulty-aware efficiency reward, AdaTIR dynamically adjusts tool budgets based on task complexity--internalizing reasoning for simple tasks while selectively invoking tools for complex tasks. Furthermore, we identify a sign reversal problem where tool penalties outweigh correctness rewards, mistakenly penalizing correct rollouts with negative advantages. To resolve this, we propose Clipped Advantage Shaping (CAS), which ensures that correctness remains the primary objective while using efficiency as a secondary constraint. Empirical results demonstrate that AdaTIR reduces tool calls by up to 97.6% on simple tasks and 28.2% on complex challenges while maintaining or enhancing accuracy. Notably, AdaTIR successfully internalizes reasoning, outperforming baselines by 4.8% on AIME 2024 even when tool access is strictly disabled.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。