用注意力分布差异检测大模型幻觉，无需重复采样。
Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals
- 通过注意力头与均匀分布的KL散度衡量不确定性
- 在多数据集上准确预测答案正确性，性能媲美现有方法
- 信号集中在中间层和事实性词元，可解释性强
我们提出一种轻量级、单次运行的不确定性量化方法，用于检测大语言模型中的幻觉。该方法利用注意力矩阵估计不确定性，无需重复采样或外部模型。具体而言，计算每个注意力头分布与均匀参考分布之间的Kullback-Leibler散度，并将其作为逻辑回归探测器的特征。在多个数据集、任务类型和模型家族中，注意力发散对答案正确性具有高度预测能力，性能可与现有不确定性估计方法相媲美。研究发现，该信号集中于中间层及命名实体、数字等事实性词元，表明注意力动态提供了高效且可解释的白盒不确定性信号。
原文摘要 · Abstract (English)
We propose a lightweight and single-pass uncertainty quantification method for detecting hallucinations in Large Language Models. The method uses attention matrices to estimate uncertainty without requiring repeated sampling or external models. Specifically, we measure the Kullback-Leibler divergence between each attention head's distribution and a uniform reference distribution, and use these features in a logistic regression probe. Across multiple datasets, task types, and model families, attention divergence is highly predictive of answer correctness and performs competitively with existing uncertainty estimation methods. We find that this signal is concentrated in middle layers and on factual tokens such as named entities and numbers, suggesting that attention dynamics provides an efficient and interpretable white-box signal of model uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。