发现大模型嵌入空间中隐藏着概念层级的几何结构。
The Lattice Representation Hypothesis of Large Language Models
- 用半空间交集构建概念格，将向量方向与逻辑运算对应。
- 实验证明词网子层级中嵌入能准确编码概念格结构。
- 适合对符号推理与神经表示统一感兴趣的读者。
我们提出大语言模型的格表示假设：一种以符号为根基的架构，将概念层级与逻辑操作嵌入到嵌入空间的几何结构中。该框架融合线性表示假设与形式概念分析（FCA），表明具有分离阈值的线性属性方向可通过半空间交集生成概念格。这种几何结构支持通过几何交（交集）和并（并集）实现符号推理，当属性方向线性无关时可得到标准形式。在WordNet子层级上的实验表明，大模型嵌入确实编码了概念格及其逻辑结构，揭示了连续几何与符号抽象之间的原理性桥梁。数据集与代码开源：https://github.com/xiongbo010/lattice-representation-hypothesis。
原文摘要 · Abstract (English)
We propose the Lattice Representation Hypothesis of large language models: a symbolic backbone that grounds conceptual hierarchies and logical operations in embedding geometry. Our framework unifies the Linear Representation Hypothesis with Formal Concept Analysis (FCA), showing that linear attribute directions with separating thresholds induce a concept lattice via half-space intersections. This geometry enables symbolic reasoning through geometric meet (intersection) and join (union) operations, and admits a canonical form when attribute directions are linearly independent. Experiments on WordNet sub-hierarchies provide empirical evidence that LLM embeddings encode concept lattices and their logical structure, revealing a principled bridge between continuous geometry and symbolic abstraction. Datasets and code are open available at https://github.com/xiongbo010/lattice-representation-hypothesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。