用单张图片生成可交互的3D场景程序，让仿真更真实。
NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulation
- 将单图重建转为可执行代码，自动定义物体形状与物理属性。
- 通过物理引擎迭代优化，确保物体组合和动作符合真实规律。
- 适合做机器人仿真、复杂操作任务的开发与测试。
Embodied AI的发展需要高质量的仿真资源以忠实还原真实世界。然而，现有单目图像转URDF的方法缺乏物理基础和场景级交互性，难以生成可用场景。本文提出NeoWorld-Pro，将单目场景重建重构为可交互3D环境的程序化编程问题。利用多模态大模型（MLLM）的零样本推理与代码生成能力，从单张RGB图像生成可执行程序，描述物体几何、运动结构及物理属性。引入“物理在回路”机制，通过在物理引擎中验证程序执行，迭代优化生成结果，确保运动结构合理、物体组合有效、空间关系准确。实验表明，NeoWorld-Pro优于开环及以往单目重建方法，在稳定堆叠、精细操作等下游任务中表现优异。
原文摘要 · Abstract (English)
The advancement of Embodied AI necessitates high-quality simulation assets that faithfully mirror the real world. However, transforming raw visual observations into simulation-ready scenes remains challenging due to the lack of physical grounding and scene-level interactivity in current image-to-URDF methods. We propose NeoWorld-Pro, a framework that reformulates monocular scene reconstruction as procedural programming for interactive 3D environments. Leveraging the zero-shot reasoning and code synthesis capabilities of MLLMs, NeoWorld-Pro converts a single RGB image into executable programs specifying object geometry, articulation, and physical properties. A physics-in-the-loop mechanism then iteratively refines the generated programs by validating their execution in a physics engine, enforcing physically plausible articulations, valid object compositions and interactions, and accurate spatial relationships. Experiments show that NeoWorld-Pro outperforms open-loop and prior monocular reconstruction methods, while enabling complex downstream tasks such as stable stacking and fine-grained manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。