发现大模型隐含线性空间世界模型，可解码物体位置。
Linear Spatial World Models Emerge in Large Language Models
- 用合成数据训练探测器解码物体位置
- 实验证明嵌入空间具几何一致性
- 适合研究模型内部表征的学者
大型语言模型在多种任务中展现出涌现能力，引发对其是否具备内部世界模型的疑问。本文研究大模型是否隐式编码线性空间世界模型，即物理空间与物体配置的线性表示。我们提出一个形式化框架来定义空间世界模型，并评估上下文嵌入中是否存在此类结构。利用物体位置的合成数据集，训练探测器以解码物体位置，并检验底层空间的几何一致性。进一步通过因果干预测试这些空间表示是否被模型实际使用。结果提供了实证证据，表明大模型确实编码了线性空间世界模型。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated emergent abilities across diverse tasks, raising the question of whether they acquire internal world models. In this work, we investigate whether LLMs implicitly encode linear spatial world models, which we define as linear representations of physical space and object configurations. We introduce a formal framework for spatial world models and assess whether such structure emerges in contextual embeddings. Using a synthetic dataset of object positions, we train probes to decode object positions and evaluate geometric consistency of the underlying space. We further conduct causal interventions to test whether these spatial representations are functionally used by the model. Our results provide empirical evidence that LLMs encode linear spatial world models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。