构建空间智能与代理能力的统一框架,推动机器人等系统在真实世界中的自主决策。
From Perception to Action: Spatial AI Agents and World Models
- 提出三维感知-推理-行动的三轴分类法,连接代理与空间任务
- 发现分层记忆对长周期空间任务至关重要,图神经网络与大模型融合前景好
- 强调世界模型在微观到宏观尺度部署中的关键作用,适合机器人和自动驾驶研究者
尽管大型语言模型在符号领域表现出色,但其在物理世界的代理推理与规划能力仍受限。空间智能——即理解三维结构、推理物体关系并受物理约束行动的能力——是具身智能的关键。现有综述仅关注代理架构或空间领域之一,缺乏统一框架。本文通过梳理2000余篇论文、引用742篇顶会文献,提出一个连接代理能力与空间任务的三轴分类体系。重点区分空间地基(几何与物理的度量理解)与符号地基(图像与文本关联),指出仅靠感知不等于具备代理能力。分析揭示三大发现:(1) 分层记忆系统(能力轴)对长时程空间任务至关重要;(2) 图神经网络与大模型融合(任务轴)是结构化空间推理的有前景路径;(3) 世界模型(尺度轴)对跨微至宏尺度的安全部署不可或缺。最后提出六个重大挑战,并呼吁建立统一评估框架以标准化跨领域评价。该分类体系为整合碎片化研究提供基础,助力下一代具空间感知能力的自主系统在机器人、自动驾驶与地理空间情报中的发展。
原文摘要 · Abstract (English)
While large language models have become the prevailing approach for agentic reasoning and planning, their success in symbolic domains does not readily translate to the physical world. Spatial intelligence, the ability to perceive 3D structure, reason about object relationships, and act under physical constraints, is an orthogonal capability that proves important for embodied agents. Existing surveys address either agentic architectures or spatial domains in isolation. None provide a unified framework connecting these complementary capabilities. This paper bridges that gap. Through a thorough review of over 2,000 papers, citing 742 works from top-tier venues, we introduce a unified three-axis taxonomy connecting agentic capabilities with spatial tasks across scales. Crucially, we distinguish spatial grounding (metric understanding of geometry and physics) from symbolic grounding (associating images with text), arguing that perception alone does not confer agency. Our analysis reveals three key findings mapped to these axes: (1) hierarchical memory systems (Capability axis) are important for long-horizon spatial tasks. (2) GNN-LLM integration (Task axis) is a promising approach for structured spatial reasoning. (3) World models (Scale axis) are essential for safe deployment across micro-to-macro spatial scales. We conclude by identifying six grand challenges and outlining directions for future research, including the need for unified evaluation frameworks to standardize cross-domain assessment. This taxonomy provides a foundation for unifying fragmented research efforts and enabling the next generation of spatially-aware autonomous systems in robotics, autonomous vehicles, and geospatial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。