让大模型自己识别错误,靠内部状态就能判断对错。
Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits
- 通过分析隐藏层和注意力模式,从内部状态提取纠错信号。
- 在数学推理等任务中,准确率超越外部判别器,且计算开销极低。
- 可零样本应用,提前发现错误生成路径,适合资源受限场景。
大型语言模型(LLMs)虽能生成流畅复杂的文本,却常无法识别自身错误与幻觉。现有方法多依赖外部评判、多样本一致性或基于文本的自我批判,带来额外计算开销或与真实正确性相关性弱。本文探讨:能否通过推断过程中的内部状态预测模型失败?我们提出 Gnosis,一种轻量级自知机制,使冻结的 LLM 能通过解码隐藏状态与注意力模式实现内在自验证。Gnosis 无须主动干预,被动观察内部轨迹,将信息压缩为固定预算的特征表示,并以极低开销预测正确性,仅增加约 500 万参数,且与序列长度无关。在数学推理、开放域问答和学术知识基准上,使用 1.7B 到 20B 参数的冻结主干模型,Gnosis 均显著优于强内部基线与大型外部评判者,在准确性和校准度上表现更佳。此外,其支持零样本泛化至部分生成,实现失败轨迹的早期检测与计算感知控制。结果表明,可靠的正确性线索内生于生成过程,可高效提取而无需外部监督。
原文摘要 · Abstract (English)
Large language models (LLMs) generate fluent and complex outputs but often fail to recognize their own mistakes and hallucinations. Existing approaches typically rely on external judges, multi-sample consistency, or text-based self-critique, which incur additional compute or correlate weakly with true correctness. We ask: can LLMs predict their own failures by inspecting internal states during inference? We introduce Gnosis, a lightweight self-awareness mechanism that enables frozen LLMs to perform intrinsic self-verification by decoding signals from hidden states and attention patterns. Gnosis passively observes internal traces, compresses them into fixed-budget descriptors, and predicts correctness with negligible inference cost, adding only ~5M parameters and operating independently of sequence length. Across math reasoning, open-domain question answering, and academic knowledge benchmarks, and over frozen backbones ranging from 1.7B to 20B parameters, Gnosis consistently outperforms strong internal baselines and large external judges in both accuracy and calibration. Moreover, it generalizes zero-shot to partial generations, enabling early detection of failing trajectories and compute-aware control. These results show that reliable correctness cues are intrinsic to generation process and can be extracted efficiently without external supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。