用注意力分布熵预测大模型答案正确性，无需人工标注。
Attention Head Entropy of LLMs Predicts Answer Correctness
- 通过计算注意力头的2-瑞尼熵，量化注意力分散程度来判断答案对错。
- 在5个模型、3个数据集上，外域泛化性能比基线平均高8.5% AUROC。
- 提问和上下文阶段的注意力模式已具备预测能力，适合安全敏感场景评估。
大型语言模型常生成看似合理却错误的答案，危及医疗等安全关键领域。人工评估成本高，而基于模型自评的方法可能引入隐藏误差。现有白盒方法通过定位注意力集中区域检测上下文幻觉，但两个问题仍未解决：能否用于预测答案正确性？是否能在跨领域场景泛化？本文提出Head Entropy，从注意力熵模式预测答案正确性，具体测量注意力质量的分散程度。利用稀疏逻辑回归分析每个注意力头的2-瑞尼熵，该方法在原域表现匹配或超越基线，在跨域泛化上显著更优，平均比最接近的基线高出8.5% AUROC。进一步发现，生成答案前仅基于问题与上下文的注意力模式，已蕴含可预测信号，此时相比基线平均提升17.7% AUROC。实验覆盖5个指令微调的大模型与3个问答数据集，涵盖通用知识、多跳推理和医学领域。
原文摘要 · Abstract (English)
Large language models (LLMs) often generate plausible yet incorrect answers, posing risks in safety-critical settings such as medicine. Human evaluation is expensive, and LLM-as-judge approaches risk introducing hidden errors. Recent white-box methods detect contextual hallucinations using model internals, focusing on the localization of the attention mass, but two questions remain open: do these approaches extend to predicting answer correctness, and do they generalize out-of-domains? We introduce Head Entropy, a method that predicts answer correctness from attention entropy patterns, specifically measuring the spread of the attention mass. Using sparse logistic regression on per-head 2-Renyi entropies, Head Entropy matches or exceeds baselines in-distribution and generalizes substantially better on out-of-domains, it outperforms the closest baseline on average by +8.5% AUROC. We further show that attention patterns over the question/context alone, before answer generation, already carry predictive signal using Head Entropy with on average +17.7% AUROC over the closest baseline. We evaluate across 5 instruction-tuned LLMs and 3 QA datasets spanning general knowledge, multi-hop reasoning, and medicine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。