发现大模型错答却自信，根源是答案太热门但错误。
Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity
- 用知识流行度分析模型为何错得自信。
- 热门错误答案让模型信心更高，平均信心从0.765降到0.254。
- 适合想改进模型置信度估计的研究者使用。
大型语言模型常以高置信度输出错误答案，但其背后成因尚不明确。本文从知识流行度视角切入，基于维基数据三元组构建实体为中心的事实性问答任务，通过问题实体流行度、答案流行度及问题-答案共现频率刻画流行度特征。研究发现两个稳定模式：第一，幻觉答案并非随机，相比正确答案更可能更流行或与问题实体关联更频繁；第二，模型置信度与生成答案的流行度强相关，即使预测错误，更流行的答案或共现更高的答案仍获得更高置信度。结果表明，流行但错误的答案可能助长模型过自信。进一步验证，引入流行度信号可有效缓解过自信问题，在六种模型和三个数据集上，错误答案平均置信度由0.765降至0.254，整体ECE从0.356降至0.050，对齐度由77.08%提升至83.72%。
原文摘要 · Abstract (English)
Large language models (LLMs) often produce incorrect answers with high confidence, yet the factors associated with such overconfidence remain insufficiently understood. We study this problem through the lens of knowledge popularity. Using entity-centric factual QA derived from Wikidata triplets, we characterize popularity through question entity popularity, answer popularity, and question-answer co-occurrence. We find two consistent patterns. First, hallucinated answers are far from random: compared with ground-truth answers, they tend to be more popular or more frequently associated with the question entity. Second, confidence is strongly tied to the popularity of generated answers: even among incorrect predictions, more popular answers or those with higher question-answer co-occurrence receive higher confidence. Together, these findings suggest that popular but wrong alternatives may contribute to overconfidence. We further show that popularity-related signals can mitigate overconfidence and improve overall confidence estimation. Across six models and three datasets, incorporating knowledge popularity reduces average confidence on incorrect answers from 0.765 to 0.254 and overall ECE from 0.356 to 0.050, while improving Alignment from 77.08% to 83.72%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。