综述3D生成如何支撑具身智能与机器人仿真,聚焦交互可用性而非单纯视觉真实。
3D Generation for Embodied AI and Robotic Simulation: A Survey

- 按数据生成、环境构建、仿真-现实桥梁三角色系统梳理3D生成技术
- 强调物理属性与可交互性,突破传统仅追求视觉逼真的局限
- 适合研究机器人仿真、数字孪生及跨域迁移学习的学者参考
具身智能与机器人系统日益依赖可扩展、多样化且具备物理基础的3D内容,用于基于仿真的训练与真实部署。尽管3D生成建模进展迅速,但具身应用对生成内容的要求远超视觉真实:生成物体需具备运动结构与材料属性,场景须支持交互与任务执行,且内容需弥合仿真与现实之间的差距。本综述围绕3D生成在具身系统中的三大作用展开:作为数据生成器,创建可用于下游交互的关节式、物理化、可变形资产;作为仿真环境构建者,生成结构感知、可控制、具有智能体特性的任务导向场景;作为仿真到现实的桥梁,支持数字孪生重建、数据增强与合成示范,以促进机器人学习与真实世界迁移。我们指出该领域正从追求视觉真实转向交互就绪,并识别出主要瓶颈:物理标注不足、几何质量与物理有效性不匹配、评估体系碎片化、以及持续存在的仿真-现实鸿沟,这些均需解决才能使3D生成成为具身智能的可靠基础。项目页面见https://3dgen4robot.github.io。
原文摘要 · Abstract (English)
Embodied AI and robotic systems increasingly depend on scalable, diverse, and physically grounded 3D content for simulation-based training and real-world deployment. While 3D generative modeling has advanced rapidly, embodied applications impose requirements far beyond visual realism: generated objects must carry kinematic structure and material properties, scenes must support interaction and task execution, and the resulting content must bridge the gap between simulation and reality. This survey reviews 3D generation for embodied AI and organizes the literature around three roles that 3D generation plays in embodied systems. In Data Generator, 3D generation produces simulation-ready objects and assets, including articulated, physically grounded, and deformable content for downstream interaction; in Simulation Environments, it constructs interactive and task-oriented worlds, spanning structure-aware, controllable, and agentic scene generation; and in Sim2Real Bridge, it supports digital twin reconstruction, data augmentation, and synthetic demonstrations for downstream robot learning and real-world transfer. We also show that the field is shifting from visual realism toward interaction readiness, and we identify the main bottlenecks, including limited physical annotations, the gap between geometric quality and physical validity, fragmented evaluation, and the persistent sim-to-real divide, that must be addressed for 3D generation to become a dependable foundation for embodied intelligence. Our project page is at https://3dgen4robot.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。