模型早知道何时该调用工具，却常误用，新方法让其更聪明地决策。
LLM Agents Already Know When to Call Tools -- Even Without Reasoning
- 通过隐藏状态分析发现模型已掌握工具调用判断能力。
- 提出Probe&Prefill方法，减少48%不必要的工具调用。
- 无需训练，适配各类大模型，兼顾效率与准确率。
工具增强型大模型代理常在无需调用工具时仍频繁调用，造成资源浪费。现有基准无法系统评估工具调用必要性。我们提出When2Tool，包含18个环境（15个单跳、3个多跳），覆盖计算规模、知识边界和执行可靠性三类工具必要性场景，具备可控难度与清晰决策边界。评估两类无训练基线：仅靠提示词（Prompt-only）和先推理再行动（Reason-then-Act）。两者均控制有限：前者抑制必要调用，后者在困难任务上精度损失严重。探查模型隐藏状态发现，工具必要性在预生成表示中可线性解码，六种模型的AUROC达0.89–0.96，远超模型自身显式推理。这表明模型已知何时需工具，但生成时未采纳该知识。据此提出Probe&Prefill：用轻量线性探测读取隐藏状态信号，并插入引导语句预填充响应。所有测试模型中，该方法降低48%工具调用，仅损失1.7%精度；最佳基线要么仅降6%调用，要么同等降调用却损失5倍精度。
原文摘要 · Abstract (English)
Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark systematically studies when a tool call is actually needed. We propose When2Tool, a benchmark of 18 environments (15 single-hop, 3 multi-hop) spanning three categories of tool necessity -- computational scale, knowledge boundaries, and execution reliability -- each with controlled difficulty levels that create a clear decision boundary between tool-necessary and tool-unnecessary tasks. We evaluate two families of training-free baselines: Prompt-only (varying the prompt to discourage unnecessary calls) and Reason-then-Act (requiring the model to reason about tool necessity before acting). Both provide limited control: Prompt-only suppresses necessary calls alongside unnecessary ones, and Reason-then-Act still incurs a disproportionate accuracy cost on hard tasks. To understand why these baselines fail, we probe the models' hidden states and find that tool necessity is linearly decodable from the pre-generation representation with AUROC 0.89--0.96 across six models, substantially exceeding the model's own verbalized reasoning. This reveals that models already know when tools are needed, but fail to act on this knowledge during generation. Building on this finding, we propose Probe&Prefill, which uses a lightweight linear probe to read the hidden-state signal and prefills the model's response with a steering sentence. Across all models tested, Probe&Prefill reduces tool calls by 48% with only 1.7% accuracy loss, while the best baseline at comparable accuracy only reduces 6% of tool calls, or achieves a similar tool call reduction but incurs a 5$\times$ higher accuracy loss. Our code is available at https://github.com/Trustworthy-ML-Lab/when2tool
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。