让大模型真正理解空间关系,用几何计算替代语言模式匹配。
From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models

- 将位置信息作为第一优先模态,直接处理空间表示而非文字描述。
- 在新基准SpatialEval上性能显著超越传统基于提示工程的模型。
- 适合需要精准空间推理的机器人、自动驾驶等应用领域。
近期大型语言模型(LLMs)看似具备空间推理能力,但这种能力主要源于对空间语言的模式匹配,而非真正的几何空间推理。由于LLMs基于离散标记运行,缺乏对连续空间表示、显式几何计算和结构化空间操作的支持。为此,我们提出首个将位置信息作为第一类模态的多模态大模型——空间语言模型(SLM),使模型在推理过程中实现几何空间推理。SLM直接作用于学习到的空间表示,而非空间关系的文字描述。为支持有效训练,我们构建了空间指令数据集(Spatial Instruction Dataset),对齐空间表示、基本几何操作与自然语言指令。我们还提出了新基准SpatialEval,用于评估属性、距离、拓扑及相对位置等空间推理任务。大量实验表明,SLM显著优于依赖提示工程或文本抽象的现有基于LLM的方法,证明了集成几何空间表示对鲁棒空间推理的益处。相关数据集、评估基准、训练代码及模型检查点见:https://github.com/chuchen2017/SLM。
原文摘要 · Abstract (English)
Recent large language models (LLMs) often appear to exhibit spatial reasoning ability; however, this capability is largely \emph{symbolic}, arising from pattern matching over spatial language rather than true \emph{geometric} reasoning over space. Because LLMs operate on discrete tokens, they lack native support for continuous spatial representations, explicit geometric computation, and structured spatial operators. To address this limitation, we introduce the \emph{Spatial Language Model (SLM)}, the first multimodal LLM that treats location information as a first-class modality and enables geometric spatial reasoning within the model's inference process. SLM directly operates on learned spatial representations rather than textual descriptions of spatial relations. To support effective training, we construct a \emph{Spatial Instruction Dataset} that aligns spatial representations, atomic geometric operations, and natural language instructions. We further propose a new benchmark named \emph{SpatialEval}, which is designed to evaluate spatial reasoning across attributes, distance, topology, and relative-position tasks. Extensive experiments show that SLM significantly outperforms existing LLM-based approaches that rely on symbolic reasoning via prompt engineering or textual abstraction, demonstrating the benefits of integrating geometric spatial representations for robust spatial reasoning. Our instruction dataset, evaluation benchmark, model training codes, and models' checkpoints can be found at: \hyperlink{https://github.com/chuchen2017/SLM}{https://github.com/chuchen2017/SLM}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。