揭示大模型错误输出的几何本质:记忆冲突与幻觉同源,皆因隐藏状态偏离记忆吸引子。
Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination

- 用吸引子几何解释模型记忆失败:事实存于记忆盆地,干扰或缺失导致状态漂移。
- 隐藏状态到最近记忆盆地的距离可精准区分正确回忆与幻觉,误拒率归零。
- 模型越大规模,幻觉越自信,但输出头会系统性抹除这种认知状态。
语言模型依赖两种知识来源:权重中编码的事实(参数化记忆,PM)和上下文信息(工作记忆,WM)。本文研究两种机制不同的失效模式——当PM与WM冲突时产生干扰;当查询事实未被学习时引发幻觉。二者均生成高置信度输出,使基于输出的监测失效。我们发现两者共享统一的几何解释:在自回归生成的隐藏状态空间中,已学事实形成吸引子盆地。冲突表现为盆地竞争:WM干扰收敛至正确盆地,却不提升输出熵;幻觉则源于盆地缺失:无记忆盆地时隐藏状态自由漂移。固定的语言模型输出头无法区分这两种情况,始终自信输出。我们在可控合成任务中验证该机制:实体标识符映射为唯一编码,通过LoRA适配器植入PM,可因果隔离各组件角色。几何间距(隐藏状态到最近记忆盆地的距离)直接读取此几何结构,其分离效果远优于输出熵,且零误拒;在自然语言事实查询上无需微调即有效,证明该几何为结构性特征而非微调产物。自信幻觉比例遵循缩放律 $C = \exp(-c/\barΔ)$,随规模增长而上升,尽管整体错误率下降。隐藏状态稳定编码认知状态,但固定输出头系统性抹除它,且该抹除随规模加剧。
原文摘要 · Abstract (English)
Language models draw on two knowledge sources: facts baked into weights (parametric memory, PM) and information in context (working memory, WM). We study two mechanistically distinct failure modes--conflict, when PM and WM disagree and interfere; and hallucination, when the queried fact was never learned. Both produce confident output regardless, making output-based monitoring blind by design. We show both failures share a unified geometric account. In the hidden-state space of autoregressive generation, learned facts form attractor basins. Conflict is basin competition: WM disrupts convergence to the correct basin without raising output entropy. Hallucination is basin absence: the hidden state drifts freely when no memorized basin exists. The frozen LM head, designed for next-token prediction, cannot distinguish these cases and fires confidently either way. We verify this account in a controlled synthetic task-entity identifiers mapped to unique codes with PM installed via LoRA adapters--where ground truth is exact and component roles can be causally isolated through targeted adapter placement. Geometric margin--the hidden state's distance to the nearest memorized basin--reads this geometry directly and separates correct recall from hallucination far more cleanly than output entropy, with zero false refusals where entropy-based detection cannot avoid rejecting the vast majority of correct outputs. The separation holds on natural-language factual queries from the pretrained model with no adaptation, confirming attractor geometry is structural rather than a fine-tuning artifact. The fraction of confident hallucinations follows a scaling law $C = \exp(-c/\barΔ)$, growing with scale even as overall error rates fall. Hidden states reliably encode epistemic state; the frozen output head systematically erases it--and this erasure worsens with scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。