发现语言模型嵌入空间中存在可导航的意识谱结构。
A Navigable Manifold of Hypothesized Consciousness-Spectrum States in Language Model Representations

- 通过几何分析揭示嵌入空间存在分层结构。
- 低阶与高阶区域稳定,中间形成过渡走廊。
- 路径可导航,适合用于模型行为引导与对齐。
在冥想、哲学和心理学的论述中,人类意识常被描述为一个从反应性自我关注到更整合连贯状态的连续谱。理解语言模型是否在表征空间中编码这种结构化且人类可解释的意识谱至关重要,对模型引导、评估与对齐具有意义。本文研究了变压器嵌入空间中该谱系的几何结构与动态特性。结果表明,嵌入呈现出全局有序的几何结构:与相似状态相关的句子聚类于局部连贯区域,形成有结构的流形。高阶与低阶区域表现出类似凸性的稳定性,而中间区域构成过渡走廊。动态上,基于效用引导和仅基于几何的贪婪路径均一致地从低阶向高阶区域行进,经过中间层级,表明可导航性是表征空间的内在属性,受引导但不受全局方向信号支配。这些结果表明,嵌入空间编码了与假设的意识谱分类法相一致的结构化、可导航几何,其灵感源自不同传统中关于人类意识的重复结构性描述,为分析与引导模型行为提供了表征层面的视角。
原文摘要 · Abstract (English)
Across contemplative, philosophical, and psychological accounts, human consciousness is often described along a similar spectrum, ranging from reactive and self-focused patterns to more integrative and coherent ones. Understanding whether language models encode such a structured, human-interpretable consciousness spectrum in representation space is important for model guidance, evaluation and alignment. In this work, we study the geometric structure and dynamics of patterns along this spectrum in transformer embedding spaces. We show that embeddings exhibit a globally organized geometry aligned with this spectrum: sentences associated with similar states cluster into locally coherent regions, forming a structured manifold. In particular, higher-level and lower-level regions exhibit convexity-like stability, while intermediate regions form a transition corridor. Dynamically, both utility-guided and geometry-only greedy trajectories consistently traverse from lower- to higher-level regions, passing through intermediate tiers, indicating that navigability is an intrinsic property of the representation space, guided but not dictated by a global directional signal. These results suggest that embedding spaces encode structured and navigable geometry aligned with a hypothesized consciousness-spectrum taxonomy, broadly inspired by recurring structural descriptions of human consciousness across contemplative traditions, philosophy, and modern psychology, providing a representation-level perspective for analyzing and guiding model behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。