用生成式AI低成本构建可交互的高保真3D物理世界,用于智能体训练与评估。
EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligence
- 基于生成式AI实现图像/文本到3D资产的自动创建
- 支持真实尺度与物理属性,输出URDF格式供仿真使用
- 适合研究机器人、强化学习等需要多样化环境的场景
构建物理上真实且精确缩放的模拟3D世界,对智能体任务的训练与评估至关重要。3D数据资产的多样性、真实感、低成本和易获取性,是实现智能体人工智能泛化与可扩展性的关键。然而,当前多数任务仍依赖手工创建和标注的传统3D图形资源,存在制作成本高、真实感不足等问题,严重制约了数据驱动方法的可扩展性。本文提出EmbodiedGen,一个面向交互式3D世界生成的基础平台。该平台可低成本生成高质量、可控、逼真的3D资产,具备真实物理属性与现实尺度,以统一机器人描述格式(URDF)输出,可直接导入各类物理仿真引擎,实现精细物理控制,支持下游任务的训练与评估。EmbodiedGen是一个包含六项核心模块的完整工具包:图像转3D、文本转3D、纹理生成、关节物体生成、场景生成与布局生成。它通过生成式AI构建多样且可交互的3D世界,解决智能体相关研究在泛化与评估方面的挑战。代码已公开于https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html。
原文摘要 · Abstract (English)
Constructing a physically realistic and accurately scaled simulated 3D world is crucial for the training and evaluation of embodied intelligence tasks. The diversity, realism, low cost accessibility and affordability of 3D data assets are critical for achieving generalization and scalability in embodied AI. However, most current embodied intelligence tasks still rely heavily on traditional 3D computer graphics assets manually created and annotated, which suffer from high production costs and limited realism. These limitations significantly hinder the scalability of data driven approaches. We present EmbodiedGen, a foundational platform for interactive 3D world generation. It enables the scalable generation of high-quality, controllable and photorealistic 3D assets with accurate physical properties and real-world scale in the Unified Robotics Description Format (URDF) at low cost. These assets can be directly imported into various physics simulation engines for fine-grained physical control, supporting downstream tasks in training and evaluation. EmbodiedGen is an easy-to-use, full-featured toolkit composed of six key modules: Image-to-3D, Text-to-3D, Texture Generation, Articulated Object Generation, Scene Generation and Layout Generation. EmbodiedGen generates diverse and interactive 3D worlds composed of generative 3D assets, leveraging generative AI to address the challenges of generalization and evaluation to the needs of embodied intelligence related research. Code is available at https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。