通过分析模型内部表示,能更准确地判断预测是否可信。
What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

- 用中间激活层训练探测器,显著提升预测校准度
- 84%情况下能预判扰动对预测的影响方向
- 发现预测结果在推理前已基本确定,可节省大量计算
微调用于预测的大语言模型虽准确但常欠校准,其思维链(CoT)推理未必反映真实依据。我们以 Eternis-Forecaster 8B 在 OpenForesight 上为例,对中间激活进行表示池化探测,结果显著改善校准性;该结论在 GLM-4.7-Flash 与 GLM-4.5-Air 上也成立。通过证据移除与干扰注入测试发现,改变提示中的关键来源会改变预测,但推理痕迹不变。相同探测器可作为“说谎检测器”:其激活值比推理链更精准跟踪行为变化,在84%的情况下可预测变化方向,即使思维链隐藏扰动影响。强制回答实验显示,预测在推理前已基本固定:一次预推理遍历即可恢复答案与置信度,按此分布路由问题可节省30%-47%生成token,且精度不变。这些结果表明,探测内部表示是校准、审计与筛选语言模型预测器的有效工具。
原文摘要 · Abstract (English)
Large language models fine-tuned for forecasting can be accurate yet poorly calibrated, and their chain-of-thought (CoT) reasoning may not faithfully reflect the evidence behind a forecast. We ask whether internal representations offer a more direct window into both. Working with Eternis-Forecaster 8B on OpenForesight, we train representation-pooling probes on intermediate activations and find they achieve substantially better calibration; a result that also holds for GLM-4.7-Flash and GLM-4.5-Air. We then assess CoT faithfulness through evidence ablation and diversionary injection: removing an influential source in the prompt often changes the model's forecast while leaving the reasoning trace untouched. The same probes function as lie detectors: their activations track behavioral shifts far better than the reasoning trace does, and they also predict the direction of change in 84% of cases, including when the CoT conceals the perturbation's influence. Finally, forced answering reveals that forecasts are largely fixed before reasoning begins: a single pre-reasoning pass recovers the committed answer and confidence, and routing questions by the spread of this pre-set answer distribution saves 30-47% of generated tokens, with no loss of accuracy. Together, these results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters and reasoning models more broadly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。