揭示大模型在结构化知识推理中幻觉的内在机制。
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
- 注意力过度聚焦于结构捷径，忽视完整上下文
- 前馈层无法锚定外部知识，导致依赖参数记忆
- 机制可泛化至多跳图与表格，适合开发检测工具
在诸多推理任务中，大语言模型（LLMs）依赖结构化外部知识（如图谱和表格），这些知识通常被线性化为序列标记表示。然而，即使具备充分知识，LLMs 仍会产生幻觉输出，其内在机制尚不明确。我们通过机制分析发现，幻觉源于系统性内部动态，而非随机噪声。首先，注意力过度集中于类似捷径的结构线索，而非均匀分布于全上下文；其次，前馈层未能有效锚定所提供知识，导致模型回退至参数化记忆。此外，结果表明幻觉始终与前馈层中的语义锚定失败相关，而注意力分配表现出更强的任务依赖性。最后，这些机制模式可推广至多跳图与表格场景，实现跨结构化知识格式的有效幻觉检测。
原文摘要 · Abstract (English)
In many reasoning tasks, large language models (LLMs) rely on structured external knowledge, such as graphs and tables, which is typically linearized into sequential token representations. However, even when sufficient knowledge is available, LLMs can still produce hallucinated outputs, and the underlying mechanisms behind such failures remain poorly understood. We investigate these mechanisms and find that hallucinations arise from systematic internal dynamics rather than random noise. First, attention disproportionately concentrates toward shortcut-like structural cues rather than distributing across the full context. Second, feed-forward representations fail to ground the provided knowledge, causing the model to revert to parametric memory. Moreover, our results indicate that hallucination is consistently associated with failures in semantic grounding within feed-forward layers, while attention allocation exhibits greater task-dependent variability. Finally, we show that these mechanistic patterns generalize beyond single-hop graphs to multi-hop and tabular settings, enabling effective hallucination detection across structured knowledge formats.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。