arXiv:2608.20569cs.AIcs.CL2026-08

测试大模型能否报告自身计算变化,发现它们全靠猜。

Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation

  • 构建干预框架,检测模型对内部改动的自我报告能力。
  • 78000次测试中,模型表现仅略高于随机(AUROC~0.5007)。
  • 信号存在但无法输出,提示报告路径有缺陷,适合关注模型可信性的研究者。

前沿大模型能否反思自身内部状态?近期研究暗示复杂模型可在特定条件下审计自身、识别变化并自信报告。我们对七家族八款开源模型进行了测试,结果表明其无法做到:当被问及自身计算是否被改变时,所有模型的表现均未超过随机水平。为此,我们构建了开放权重掩码内省(OWMI)框架,通过干预残差流位置、注意力头和稀疏自编码器特征,并对比真实干预与无更改、匹配影响的随机扰动及仅见输出的文本观察者。在超过78,000次测量中,没有任何模型能将真实干预与虚假情况区分,平均受试者工作特征曲线下面积(AUROC)约为0.5007,等效性检验将效果上限控制在0.15个百分点以内。令人意外的是,相关信息其实已存在于模型内部——经微调的模型可近乎完美恢复干预方向,线性探测器在激活值上实现75%至95.8%准确率,且最后一层前无遗漏错误。在某一模型中,信号仅体现在置信度而非回答内容上:其是/否回答恒定不变,但置信度可将干预与假阳性分离,达到AUROC 0.647。失败出在从内部状态到语言输出的路径上,因此依赖模型自述的监督机制必须用内部参照进行验证。尽管当前开源模型不具备内省能力,未来模型仍可能突破此限制。

原文摘要 · Abstract (English)

Are frontier models able to introspect about their internal states? Recent work suggests that under certain conditions a complex enough model can audit its own internals, call out what changed, and report back confidently about it. We tested that claim on eight open-weight models from seven families and found no such ability: asked whether their own computation had been altered, none answered better than chance. To test it we built Open-Weight Masked Introspection (OWMI), a framework that intervenes on residual-stream sites, attention heads and sparse-autoencoder features, then interrogates the model about the change against the null conditions an answer has to beat: sham runs where nothing was altered, impact-matched random perturbations, and a text-only observer that sees only the visible output. Over 78,000 measurements, no model's report discriminates a real intervention from a sham beyond chance (AUROC ~0.5007), and an equivalence test bounds the effect below 0.15 percentage points of AUROC. Surprisingly, all the information needed is in the models. A model fine-tuned to report this class of intervention reaches near-perfect recovery on held-out directions, and a linear probe recovers intervention presence from the same activations at 75% to 95.8% accuracy, sharpening to no held-out error at the last layer before the model speaks. In one model the signal surfaces in the confidence rather than the words: its yes-or-no report never varies, while the confidence attached to it separates intervention from sham at AUROC 0.647. The failure sits in the path from internal state to verbal report, so oversight that reads a model's own testimony needs validating against an internal reference. While our results show the inability of current open-weight models to introspect, the debate is not settled for future models.

模型内省大模型评估可信报告神经机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。