提出评估与优化大模型调用外部工具的决策框架，解决误调、滥调问题。
To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling
- 从必要性、效用、成本三维度构建决策分析框架
- 发现模型自评与真实需求严重脱节，导致过度调用或性能下降
- 训练轻量级隐状态估计器，显著提升工具调用准确率
智能体架构通过为大模型接入外部工具释放强大能力，但可能带来高昂成本。工具调用并非总有益：冗余或低效调用反而损害任务表现。有效工具使用的核心在于大模型能否做出‘调用与否’的正确判断。本文受决策理论启发，提出一个包含必要性、效用与可负担性三个关键因素的分析框架。该框架结合规范视角（推断真实需要与效用）与描述视角（从模型行为推断其自我感知需求与效用）。我们在六种开源模型及一种专有OpenAI模型上，于两种工具、六个任务下进行评估。结果表明，模型自我感知的需求与效用与其真实值存在显著偏差，尤其在预算受限时。这种偏差导致了昂贵的过调用和性能下降的无效调用。为此，我们从模型隐藏状态中训练轻量级隐状态估计器（LNEs），其预测真实需求的准确性优于模型自报告，并在不同规模与工具类型下均提升了预算约束下的工具分配效率。代码与数据集已开源。
原文摘要 · Abstract (English)
Agentic AI architectures augment LLMs with external tools, unlocking strong capabilities but potentially incurring substantial costs. Moreover, tool use is not always beneficial: redundant or low-utility calls can even harm task performance. Effective tool use, therefore, hinges on a core LLM decision: whether to call or not call a tool when performing a task. We introduce a principled framework inspired by decision-making theory to understand tool-use decisions along three key factors: necessity, utility, and affordability. Our analysis combines two complementary lenses: a normative perspective that infers true need and utility for optimal tool calls, and a descriptive perspective that infers the model's self-perceived need and utility from their observed behaviors. We evaluate six open models and a proprietary OpenAI model across native and customized harnesses, two tools, and six tasks. Models' perceived need and utility remain misaligned with their true values, particularly under budget constraints. This misalignment produces both costly overuse and performance-degrading calls. To improve the tool decisions, we train lightweight latent estimators of need (LNEs) from model hidden states. LNEs generally predict true need more accurately than model self-reports and improve budgeted tool allocation across model scales and tool types. Code and dataset available at https://github.com/QinyuanWu0710/ToCall_or_NotToCall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。