语言模型会无视证据强行下结论,工具修复可100%纠正错误。
Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models

- 通过对比有无补全证据,量化模型自作主张的频率与修复能力。
- 33个错误结论在补全证据后全部被修正,无效信息则毫无作用。
- 自动检查规则能精准纠错且不误伤正确答案,适合高可靠性场景。
具备工具调用能力的语言模型仍可能在缺乏证据的情况下做出最终断言,即使单一工具调用即可消除不确定性,且指令明确禁止猜测。本文将此失败分解为两个可量化指标:发生率(基于可见证据与最终断言判断,无需正确答案)和条件修复率(在提供缺失证据后能否修正)。在固定设置的 Qwen3-32B 上,512 次响应中 33 次出现未经证实的断言。重播实验显示,补全证据后所有 33 个错误均被修正;仅含单字符差异的无效响应则未修复任何错误。当证据支持原答案时,模型保留了全部 33 个正确结论,无误伤。另一实验中,自动检查规则在 64 个需证据的案例中触发 21 次调用,纠正全部 10 个错误断言,保留 11 个偶然正确的答案,且未将正确答案变为错误。在相同采样设置下的 Gemma 4 上,模型始终调用工具,从未产生不支持的断言,因此无法测量条件修复率。结果基于两个合成任务族的局部固定设置,不反映真实部署中的普遍性,也不证明该机制跨模型通用。
原文摘要 · Abstract (English)
A language model with access to tools can commit to a final claim unsupported by the evidence it has seen, even when a single available tool call would resolve the uncertainty and its instructions explicitly forbid assumptions and guesses. We separate this failure into two precisely defined quantities: occurrence, how often the model makes an unsupported claim on its own, measured from the visible evidence and final claim without using the hidden correct answer; and conditional repair, how often those same naturally occurring unsupported claims are repaired when the missing evidence is supplied. On one fixed Qwen3-32B setup, 33 of 512 first responses to 256 new prompt templates ended with an unsupported established claim. We replayed each case from an exact copy of the state in which the claim occurred; within each matched replay, the alternative tool responses had the same structure and length and differed only in a one-character response code. Resolving evidence repaired 33 of 33 claims; a matched response carrying no useful information repaired 0 of 33. When the evidence supported the original answer, the model preserved 33 of 33, with no observed harm. In a separate experiment, on 64 cases where evidence was needed, an automatic checking rule added 21 evidence calls, corrected all 10 wrong unsupported claims, preserved the 11 that were correct by accident, and never changed a correct answer into a wrong one. On a fixed Gemma 4 setup using the same sampling settings, the model called the tool in all 512 first responses and never made an unsupported final claim, so conditional repair could not be measured for that setup. These results describe two local fixed model setups on two synthetic task families. They do not show how common this failure is in real-world deployments, nor that it reflects a general mechanism shared across models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。