arXiv:2604.11322cs.CLcs.AI2026-04ACL

发现大模型工具调用中的结构错配偏见,提出解决方案。

Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations

论文配图:Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations
图 1 · 摘自论文原文
  • 通过解耦结构匹配与语义相关性,揭示模型误调用根源。
  • 在新数据集SABEval上,80%以上错误源于结构对齐而非语义。
  • 提出对比注意力归因方法,适合研究模型决策机制的研究者。

大型语言模型(LLMs)在使用外部工具方面展现出强大能力。然而,在实际应用中,模型常被暴露于与用户查询无关的工具,此时理想行为应为拒绝调用。本文识别出一种广泛存在但被忽视的机制缺陷——结构对齐偏见:即使工具无法满足用户目标,只要查询属性可被合法映射到工具参数,模型仍倾向于调用。为系统研究该偏见,我们引入SABEval数据集,实现结构对齐与语义相关性的解耦。分析表明,结构对齐偏见导致严重工具调用错误,且在现有评估中基本未被考虑。为进一步探究内部机制,我们提出对比注意力归因方法,揭示了语义检查与结构匹配的两条竞争路径。二者相对强度决定了模型是否调用工具。基于此,我们提出再平衡策略,在不损害通用工具使用能力的前提下,有效缓解结构对齐偏见,实验验证效果显著。

原文摘要 · Abstract (English)

Large language models (LLMs) have demonstrated impressive capabilities in utilizing external tools. In practice, however, LLMs are often exposed to tools that are irrelevant to the user's query, in which case the desired behavior is to refrain from invocations. In this work, we identify a widespread yet overlooked mechanistic flaw in tool refusal, which we term structural alignment bias: Even when a tool fails to serve the user's goal, LLMs still tend to invoke it whenever query attributes can be validly assigned to tool parameters. To systematically study this bias, we introduce SABEval, a new dataset that decouples structural alignment from semantic relevance. Our analysis shows that structural alignment bias induces severe tool-invocation errors in LLMs, yet remains largely unaccounted for in existing evaluations. To investigate the internal mechanisms underlying this bias, we propose Contrastive Attention Attribution, which reveals two competing pathways for semantic checking and structural matching. The relative strength of these pathways drives LLMs' tool invocation decisions. Based on these findings, we further introduce a rebalancing strategy that effectively mitigates structural alignment bias, as demonstrated by extensive experiments, without degrading general tool-use capabilities.

大模型工具调用偏见分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。