arXiv:2604.15741cs.CLcs.AI2026-04ACL

通过分析模型内部表示的分散度,有效检测大模型幻觉。

Learning Uncertainty from Sequential Internal Dispersion in Large Language Models

论文配图:Learning Uncertainty from Sequential Internal Dispersion in Large Language Models
图 1 · 摘自论文原文
  • 基于每层每词的隐藏状态分散度建模不确定性
  • 在多个数据集上优于现有基线方法,且泛化性强
  • 无需大量标注数据,适合实际部署

不确定性估计是检测大语言模型幻觉的有前景方法。现有方法通常依赖模型内部状态估计不确定性,但受限于对隐藏状态跨层演化的严格假设,且仅关注最后一层或平均词元导致信息丢失。为此,我们提出序列内部方差表示(SIVR),一种监督式幻觉检测框架,利用从隐藏状态中提取的逐词、逐层特征。SIVR采用更基础的假设:不确定性体现为内部表示在各层间的分散程度,而非依赖特定演化规律,因此具备模型和任务无关性。同时,它整合了全序列的词级方差特征,学习时间模式以识别事实错误,避免信息损失。实验表明,SIVR持续优于强基线,尤其在泛化能力上表现突出,且无需大规模训练数据,展现出实际部署潜力。代码已开源:https://github.com/ponhvoan/internal-variance。

原文摘要 · Abstract (English)

Uncertainty estimation is a promising approach to detect hallucinations in large language models (LLMs). Recent approaches commonly depend on model internal states to estimate uncertainty. However, they suffer from strict assumptions on how hidden states should evolve across layers, and from information loss by solely focusing on last or mean tokens. To address these issues, we present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework that leverages token-wise, layer-wise features derived from hidden states. SIVR adopts a more basic assumption that uncertainty manifests in the degree of dispersion or variance of internal representations across layers, rather than relying on specific assumptions, which makes the method model and task agnostic. It additionally aggregates the full sequence of per-token variance features, learning temporal patterns indicative of factual errors and thereby preventing information loss. Experimental results demonstrate SIVR consistently outperforms strong baselines. Most importantly, SIVR enjoys stronger generalisation and avoids relying on large training sets, highlighting the potential for practical deployment. Our code repository is available online at https://github.com/ponhvoan/internal-variance.

幻觉检测不确定性LLM特征分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。