梳理从2D到3D世界模型的演进,揭示三维认知的核心技术与应用前景。
From 2D to 3D Cognition: A Brief Survey of General World Models
- 构建3D世界模型的两大支柱:三维表征与世界知识融合
- 实现三维物理场景生成、空间推理与交互三大认知能力
- 适合关注具身智能、自动驾驶与数字孪生的研究者阅读
世界模型在人工智能通用化发展中日益受到重视,作为学习外部世界表征并预测未来状态的计算框架。早期研究集中于2D视觉感知与仿真,而近年的3D感知生成模型已能合成几何一致、可交互的3D环境,标志着向3D空间认知的转变。尽管进展迅速,该领域仍缺乏系统性分析以分类新兴技术及其在推进3D认知世界模型中的作用。本综述提出一个概念框架,对从2D感知向3D认知演进的世界模型进行结构化、前瞻性回顾。文中强调3D表征与世界知识融入为两大核心技术驱动力,并剖析支撑3D世界建模的三项核心认知能力:3D物理场景生成、3D空间推理与3D空间交互。进一步探讨这些能力在具身智能、自动驾驶、数字孪生及游戏/虚拟现实等真实场景中的应用。最后,识别数据、建模与部署层面的挑战,并展望更鲁棒、更通用的3D世界模型未来方向。
原文摘要 · Abstract (English)
World models have garnered increasing attention in the development of artificial general intelligence (AGI), serving as computational frameworks for learning representations of the external world and forecasting future states. While early efforts focused on 2D visual perception and simulation, recent 3D-aware generative world models have demonstrated the ability to synthesize geometrically consistent, interactive 3D environments, marking a shift toward 3D spatial cognition. Despite rapid progress, the field lacks systematic analysis to categorize emerging techniques and clarify their roles in advancing 3D cognitive world models. This survey addresses this need by introducing a conceptual framework, providing a structured and forward-looking review of world models transitioning from 2D perception to 3D cognition. Within this framework, we highlight two key technological drivers, particularly advances in 3D representations and the incorporation of world knowledge, as fundamental pillars. Building on these, we dissect three core cognitive capabilities that underpin 3D world modeling: 3D physical scene generation, 3D spatial reasoning, and 3D spatial interaction. We further examine the deployment of these capabilities in real-world applications, including embodied AI, autonomous driving, digital twin, and gaming/VR. Finally, we identify challenges across data, modeling, and deployment, and outline future directions for advancing more robust and generalizable 3D world models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。