arXiv:2606.16364cs.AIcs.CR2026-06被引 2

大模型选错工具,不是看不清,而是看对了却选错了。

Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents

  • 通过注意力分析发现,模型常正确关注目标工具
  • 修复读出环节可挽救近90%的失败案例
  • 无需训练的新方法能显著提升工具选择准确率

LLM代理在工具选择中出错,传统观点认为是因工具太多导致遗漏。本文从注意力机制切入,发现真实失败案例中,模型在80%情况下已正确关注到目标工具(随机概率仅21%),而正确答案段落反而仅被关注10%。这说明错误发生在决策输出阶段而非输入感知。实验验证:仅修复提示词只能恢复≤23%错误,而读出端干预可恢复59%-91%;不同表示空间中的干预(注意力偏置、残差流向量)效果高度一致(任务间杰卡德相似度0.79-0.91),表明瓶颈在读出层。此外,基于注意力的无训练选择器在BFCL上使函数名选择准确率提升11.9分(接近17.9分的最优差距),在Seal-Tools上提升14.9分,所有模型均显著有效(每项McNemar检验p≤8e-4)。剂量响应实验显示因果偏置具有双向单调性,但全模型跨度仅支持相关性诊断;该方法目前适用于单轮场景,尚未验证多轮泛化能力。

原文摘要 · Abstract (English)

LLM agents mis-call tools, and the natural guess is that the model failed to see the right tool in a crowded harness. We show the opposite through a lens concurrent work sets aside -- the model's attention to labeled tool-definition segments. On real BFCL failures, by per-candidate attention argmax the model attends most to the correct tool 80% of the time (vs. 21% chance), and the gold is the under-attended segment on only 10%: it looks at the right tool and still picks wrong. This directly refutes the intuitive "crowded-harness / lost-in-the-middle" explanation: the failure is at the decision readout, not the harness, and we pin it there three ways. (1) Input vs. readout: repairing the prompt (reordering or duplicating the gold tool) recovers <=23% of failures, while readout-side interventions recover 59-91%. (2) Representation-invariance: two gold-pointed interventions in different representations -- an additive attention-logit bias and a residual-stream steering vector -- recover largely the same failures (per-task Jaccard 0.865 pooled, 0.79-0.91 per model), so the bottleneck is localized to the readout independent of which representation is poked. (3) A training-free, gold-free selector: per-segment attention closes most of the gold-free-vs-oracle gap on BFCL (+11.9 pts pooled function-name selection vs. +17.9-pt oracle headroom) and adds +14.9 pts on Seal-Tools; every model positive (exact McNemar p<=8e-4 each). Scopes differ: the causal attention-bias dose-response is bidirectional and monotonic on 10 mask-honoring models (3-32B), the full 0.5-32B span carrying only the correlational diagnostic; the deployable selector is evaluated on 5 single-turn models and does not yet transfer to a multi-turn loop.

大模型工具选择注意力机制决策错误

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。