模型知道出错却说不清，探针检测不准最终答案正确性。
The Knowing-Saying Gap: When Probes See Errors that Confidence Misses
- 用线性探针检测语言模型上下文错误，准确率接近完美。
- 探针能发现错误，但无法预测最终答案是否正确，存在认知与表达分离。
- 不同模型和错误类型需定制监控策略，无通用最优干预方法。
线性探针在多跳算术推理链中可近乎完美地检测到上下文污染，但这一能力并未转化为对最终答案正确性的可靠预测。实验表明，探针检测到的错误与最终答案正确性无关；强制模型采用结构化置信度格式后，其输出仅剩两个值，且错误率无法区分；探针在推理链中的持续性也无法区分正确与错误结果，推翻了预先注册的“持续性优于峰值”假设。该现象在包括推理模型在内的多个模型家族中普遍存在。作为实时监控手段，基于探针的干预措施效果高度依赖模型和错误类型：branch-and-pick 在多数模型上表现净收益，且在 Llama-3.1-8B 上唯一不破坏正确路径（4 次挽救，0 次破坏）；而 reprompt 与 replace-prior 在挽救错误的同时，以相近比例破坏正确路径。探针监控是口头置信度的必要补充，但无单一干预方案占优，部署应采用模型感知、错误类型感知的路由策略。
原文摘要 · Abstract (English)
Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction. The result is a dissociation with direct implications for deployment monitoring. Across multi-hop arithmetic chains, probes that detect corruption turn out to be uninformative about final answer correctness; models forced into structured confidence formats collapse to two values with indistinguishable error rates; and probe persistence across hops fails to separate correct from incorrect outcomes, refuting our pre-registered "persistence beats peak" hypothesis. This pattern of knowing but not saying generalises across model families including reasoning models. As a real-time monitor, probe-based interventions are sharply model and error-type dependent: branch-and-pick is net-positive across models and uniquely non-breaking on Llama-3.1-8B (4 rescued, 0 broken), while reprompt and replace-prior break correct traces at roughly the rate they rescue wrong ones. Probe-based monitoring is a necessary complement to verbalised confidence, but no single intervention dominates, and the deployable answer is model-aware, error-type-aware routing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。