arXiv:2510.09033cs.CL2025-10ACL被引 3

大模型所谓的'知道自己不知道',其实更多是记性好而非判断真伪。

Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness

  • 区分虚假输出是源于无依据编造还是错误关联
  • 有错误关联的幻觉与真实回答隐藏状态高度相似
  • 现有检测方法对常见幻觉无效,仅能识别无依据编造

近期研究认为大模型能区分‘知道’与‘不知道’,通过内部信号辨别幻觉和正确输出。但幻觉成因复杂,除知识缺失外,还来自预训练中习得的统计捷径或虚假关联。本文提出新分类:无关联幻觉(UHs)缺乏参数支撑,关联幻觉(AHs)由虚假关联驱动。机制分析显示,隐藏状态主要反映模型是否在调用参数化知识,而非输出真实性。结果表明,AHs的隐藏状态几何结构与真实输出高度重叠,使常规检测失效;而UHs具有独特聚类特征,易于识别。

原文摘要 · Abstract (English)

Recent work suggests that LLMs "know what they don't know", positing that hallucinated and factually correct outputs arise from distinct internal processes and can therefore be distinguished using internal signals. However, hallucinations have multifaceted causes: beyond simple knowledge gaps, they can emerge from training incentives that encourage models to exploit statistical shortcuts or spurious associations learned during pretraining. In this paper, we argue that when LLMs rely on such learned associations to produce hallucinations, their internal processes are mechanistically similar to those of factual recall, as both stem from strong statistical correlations encoded in the model's parameters. To verify this, we propose a novel taxonomy categorizing hallucinations into Unassociated Hallucinations (UHs), where outputs lack parametric grounding, and Associated Hallucinations (AHs), which are driven by spurious associations. Through mechanistic analysis, we compare their computational processes and hidden-state geometries with factually correct outputs. Our results show that hidden states primarily reflect whether the model is recalling parametric knowledge rather than the truthfulness of the output itself. Consequently, AHs exhibit hidden-state geometries that largely overlap with factual outputs, rendering standard detection methods ineffective. In contrast, UHs exhibit distinctive, clustered representations that facilitate reliable detection.

大模型幻觉隐藏状态真相检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。