发现大模型事实检索是分散且冗余的，不依赖连续层计算。
Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process
- 通过追踪属性计算路径，定位最小必要层集。
- 路径常跳过层数，同一事实有多条等效路径。
- 揭示知识存储高度分布化，适合研究模型内表示机制者阅读。
大型语言模型（LLMs）存储并回忆事实知识，但实体表示如何被转换以实现特定属性检索的精确机制仍不清楚。本文从‘属性计算路径’——即生成目标属性所需的实体表示上的计算步骤序列——视角出发进行研究，并提出一种迭代修补协议，以识别完成该计算所需的最小层集。在 LLaMA 3.1 8B 和 Qwen3 8B 上应用该方法，发现这些路径具有非连续性，常跳过多个层，且对同一实体与事实存在多条功能等价路径，表明属性计算具有高度冗余性。这说明知识计算高度分布，可能解释了定位编辑不匹配现象，提示当前对大模型中知识存储与检索的理解仍极为有限。
原文摘要 · Abstract (English)
Large language models (LLMs) store and recall factual knowledge, yet the precise mechanism of how entity representations are transformed to enable specific attribute retrieval remains underexplored. In this work, we investigate this mechanism through the lens of an "attribute-computation path"-a sequence of computational steps over the entity representation required to elicit a target attribute. We then propose an iterative patching protocol to identify a minimal subset of layers necessary for this computation. Applying our method to LLaMA 3.1 8B and Qwen3 8B, we find that these paths are non-contiguous, often skipping layers, and that models possess multiple, functionally-equivalent paths for the same entity and fact, highlighting a high degree of redundancy in attribute computation. This implies that knowledge computation is highly distributed, potentially explaining the localization-editing mismatch and suggesting that knowledge storage and retrieval in LLMs is far from being well understood.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。