用层间信息一致性评估大模型不确定性，又准又省资源。
Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores
- 通过分析层间内部表示的一致性模式做不确定性估计
- 跨数据集迁移时比传统方法提升最高2.86 AUPRC
- 适合需要轻量级、可迁移不确定性的研究者使用
大型语言模型常会自信出错，可靠的不确定性估计（UE）至关重要。基于输出的启发式方法成本低但脆弱，而探测内部表示虽有效却维度高且难以迁移。本文提出一种轻量级、针对每个实例的不确定性估计方法，仅需一次前向传播即可计算层间表示的一致性得分。在三种模型上，该方法在分布内表现与探测相当，平均对角差异不超过-1.8 AUPRC百分点和+4.9 Brier分数点。在跨数据集迁移时，始终优于探测方法，非对角收益最高达+2.86 AUPRC和+21.02 Brier分数点。在4比特权重量化下仍保持鲁棒性，平均优于探测方法+1.94 AUPRC和+5.33 Brier分数点。此外，分析特定层-层交互揭示了不同模型编码不确定性的差异。整体而言，该方法提供了一种轻量、紧凑且可迁移的不确定性捕捉方式。
原文摘要 · Abstract (English)
Large language models (LLMs) are often confidently wrong, making reliable uncertainty estimation (UE) essential. Output-based heuristics are cheap but brittle, while probing internal representations is effective yet high-dimensional and hard to transfer. We propose a compact, per-instance UE method that scores cross-layer agreement patterns in internal representations using a single forward pass. Across three models, our method matches probing in-distribution, with mean diagonal differences of at most $-1.8$ AUPRC percentage points and $+4.9$ Brier score points. Under cross-dataset transfer, it consistently outperforms probing, achieving off-diagonal gains up to $+2.86$ AUPRC and $+21.02$ Brier points. Under 4-bit weight-only quantization, it remains robust, improving over probing by $+1.94$ AUPRC points and $+5.33$ Brier points on average. Beyond performance, examining specific layer--layer interactions reveals differences in how disparate models encode uncertainty. Altogether, our UE method offers a lightweight, compact means to capture transferable uncertainty in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。