arXiv:2607.08046cs.CLcs.AI2026-07被引 1

通过分析模型内部表示,能更准确地判断预测是否可信。

What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

论文配图:What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness
图 1 · 摘自论文原文
  • 用中间激活层训练探测器,显著提升预测校准度
  • 84%情况下能预判扰动对预测的影响方向
  • 发现预测结果在推理前已基本确定,可节省大量计算

微调用于预测的大语言模型虽准确但常欠校准,其思维链(CoT)推理未必反映真实依据。我们以 Eternis-Forecaster 8B 在 OpenForesight 上为例,对中间激活进行表示池化探测,结果显著改善校准性;该结论在 GLM-4.7-Flash 与 GLM-4.5-Air 上也成立。通过证据移除与干扰注入测试发现,改变提示中的关键来源会改变预测,但推理痕迹不变。相同探测器可作为“说谎检测器”:其激活值比推理链更精准跟踪行为变化,在84%的情况下可预测变化方向,即使思维链隐藏扰动影响。强制回答实验显示,预测在推理前已基本固定:一次预推理遍历即可恢复答案与置信度,按此分布路由问题可节省30%-47%生成token,且精度不变。这些结果表明,探测内部表示是校准、审计与筛选语言模型预测器的有效工具。

原文摘要 · Abstract (English)

Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-Air. We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the prompt often changes the model's forecast while leaving the reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation's influence. Finally, forced answering reveals that forecasts are largely fixed before reasoning begins: a single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly.

模型校准推理审计内部表示预测可信度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。