用AI自动生成可直接用于训练的3D仿真环境
SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
- 通过智能体迭代推理,自动构建符合任务意图的3D场景
- 生成的10000个场景在物理合理性与视觉真实度上表现优异
- 适合需要大规模仿真训练的具身智能研究者使用
现实世界中具身智能体的数据采集成本高且存在安全隐患,亟需可扩展、逼真且适配模拟器的3D环境。现有场景生成系统多依赖规则或任务特定流程,常产生伪影和物理不合理的场景。我们提出SAGE,一种智能体框架,能根据用户指定的具身任务(如“拿起碗并放到桌上”)理解意图,并自动规模化生成模拟器就绪的场景。该智能体结合布局与物体组合生成器,以及评估语义合理性、视觉真实性和物理稳定性的判别器,通过迭代推理与自适应工具选择,持续优化场景直至满足用户意图与物理有效性。生成的环境具有高度真实感与多样性,可直接部署于现代模拟器进行策略训练。仅基于该数据训练的策略展现出明确的缩放趋势,并能泛化至未见过的物体与布局,证明了仿真驱动的具身智能规模化前景。代码、演示及SAGE-10k数据集详见项目页面:https://research.nvidia.com/labs/dir/sage/
原文摘要 · Abstract (English)
Real-world data collection for embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified embodied task (e.g., "pick up a bowl and place it on the table"), understands the intent and automatically generates simulation-ready environments at scale. The agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for embodied AI. Code, demos, and the SAGE-10k dataset can be found on the project page here: https://research.nvidia.com/labs/dir/sage/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。