通过融合全局与局部熵，提升大模型幻觉检测能力。
Integrating Local and Global Entropy for Uncertainty Quantification in LLMs
- 用隐藏层几何复杂度衡量全局不确定性，令牌级熵为局部指标。
- 全局熵能捕捉局部信号忽略的自信错误模式，提升可靠性预测。
- GLU方法仅需一次前向传播，适配多种模型和任务。
大语言模型常以高置信度产生幻觉，因此不确定性量化对可靠部署至关重要。现有方法主要依赖令牌级信号，忽视了中间隐藏状态的几何结构。本文将隐藏状态矩阵的几何复杂度作为大模型全局不确定性的度量，同时将令牌级不确定性视为局部指标。我们发现，隐藏状态几何熵（全局不确定性）与令牌级熵（局部不确定性）在统计上近乎正交，分别捕捉不同的失效模式，其中全局几何结构能恢复局部信号系统性遗漏的‘自信但错误’情形。基于此，我们提出无需监督、单次前向传播的全局-局部不确定性（GLU）评分，通过乘法门控融合两者信号。在三个模型家族和六个基准测试中，GLU表现优于或持平所有无监督基线，且保持长度归一化和架构无关性。
原文摘要 · Abstract (English)
Large language models hallucinate confidently, making uncertainty quantification (UQ) essential for reliable deployment. Existing methods rely predominantly on token-level signals, leaving the geometric structure of intermediate hidden states underused. In this paper, we take the geometric complexity of hidden-state matrices as a measure of the global uncertainty of LLMs, while treating token-level uncertainty estimation as a local metric. We show that hidden-state geometric entropy (global uncertainty) and token-level entropy (local uncertainty) are statistically near-orthogonal, capturing distinct failure regimes for reliability prediction. In particular, global geometry recovers the confident-but-wrong failure mode that local signals systematically miss. Building on this, we propose Global-Local Uncertainty (GLU), an unsupervised, single-pass score that fuses the two signals via a multiplicative gate. Across three model families and six benchmarks, GLU matches or outperforms all unsupervised baselines while requiring only a single forward pass and remaining length-normalized and architecture-agnostic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。