arXiv:2608.17950cs.CL2026-08

发现大模型深层隐藏空间具六度分离特性,可揭示推理路径与幻觉本质。

Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds

论文配图:Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
图 1 · 摘自论文原文
  • 直接分析隐藏状态几何结构,绕过注意力权重干扰。
  • 深层语义层实现概念距离压缩,路径长度严格控制在6跳内。
  • 可用于零样本幻觉检测,识别生成内容是否忠实于上下文。

大型语言模型(LLMs)展现出强大的长上下文多跳推理能力,但其内部机制仍不清晰。传统基于注意力的可解释性方法常受路由伪影(如注意力黑洞)影响,难以捕捉真实语义接近性。本文摒弃注意力权重,直接分析隐藏状态流形的动态几何结构,证明深层LLM隐空间天然形成小世界网络。通过将长上下文表征的连续相似矩阵稀疏化为无权图,我们在两种不同架构中追踪了高度分离的语义锚点间的连接性。结果揭示显著的拓扑相变:早期句法层完全断裂,而深层推理层突然将巨大概念距离压缩为可导航路径,且路径长度严格受限于“六度分离”(≤6个语义跳数)。此外,我们利用该框架在RAGognize数据集上进行零样本幻觉检测,结果显示事实性生成与其源上下文保持结构完整(约3跳),而幻觉则导致严重拓扑坍塌。本工作从数学上形式化了Transformer执行抽象推理的方式,并提供了一种严格的几何标志用于评估生成内容的真实性。

原文摘要 · Abstract (English)

Large Language Models (LLMs) demonstrate remarkable multi-hop reasoning capabilities over long contexts, yet the internal mechanisms enabling these distant cognitive leaps remain poorly understood. Traditional attention-based interpretability often fails to capture true semantic proximity due to routing artifacts like attention sinks. In this paper, we bypass attention weights to directly analyze the dynamic geometry of the hidden state manifold, proving that deep LLM latent spaces natively organize into Small-World networks. By sparsifying the continuous similarity matrices of long-context representations into unweighted graphs, we trace the connectivity between highly disjoint semantic anchors across two distinct architectures. Our findings reveal a sharp topological phase transition: while early syntactic layers remain entirely fractured, deep reasoning layers abruptly compress massive conceptual distances into highly navigable pathways strictly bounded by the "Six Degrees of Separation" limit (=< 6 semantic hops). Furthermore, we demonstrate the practical efficacy of this framework by applying it to zero-shot hallucination detection within Retrieval-Augmented Generation (RAG) using the RAGognize dataset. We show that factually grounded generations maintain structural integrity with their source context (approximately 3 hops), whereas hallucinations induce severe topological collapse. Ultimately, this work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability.

大模型推理小世界网络幻觉检测几何分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。