让代码大模型生成有物理规律的4D动态世界
Code2Worlds: Empowering Coding LLMs for 4D World Generation
- 用双流架构分离物体生成与环境布局，解决多尺度混淆问题
- 通过闭环机制提升动态真实性，4D生成质量比基线高41%和49%
- 适合想做物理仿真、智能场景生成的研究者和开发者
实现空间智能需超越视觉合理性，构建基于物理规律的世界模拟器。尽管代码大模型已能生成静态3D场景，但将其扩展到4D动态生成仍是关键挑战。主要难点在于：多尺度上下文纠缠，即整体生成难以兼顾局部物体结构与全局环境布局；以及语义-物理执行差距，开环代码生成常导致缺乏动态真实性的物理幻觉。我们提出Code2Worlds框架，将4D生成任务转化为语言到模拟代码的生成。首先，设计双流架构，分离检索增强的物体生成与分层环境编排。其次，引入物理感知闭环机制：由后处理代理编写动态逻辑，并通过视觉-语言-运动批评家进行自我反思，迭代优化模拟代码。在Code4D基准上的评估显示，Code2Worlds相比基线提升41%的SGS得分和49%的丰富度，且唯一实现了具备物理意识的动态生成，而此前方法仅支持静态场景。
原文摘要 · Abstract (English)
Achieving spatial intelligence requires moving beyond visual plausibility to build world simulators grounded in physical laws. While coding LLMs have advanced static 3D scene generation, extending this paradigm to 4D dynamics remains a critical frontier. This task presents two fundamental challenges: multi-scale context entanglement, where monolithic generation fails to balance local object structures with global environmental layouts; and a semantic-physical execution gap, where open-loop code generation leads to physical hallucinations lacking dynamic fidelity. We introduce Code2Worlds, a framework that formulates 4D generation as language-to-simulation code generation. First, we propose a dual-stream architecture that disentangles retrieval-augmented object generation from hierarchical environmental orchestration. Second, to ensure dynamic fidelity, we establish a physics-aware closed-loop mechanism in which a PostProcess Agent scripts dynamics, coupled with a VLM-Motion Critic that performs self-reflection to iteratively refine simulation code. Evaluations on the Code4D benchmark show Code2Worlds outperforms baselines with a 41% SGS gain and 49% higher Richness, while uniquely generating physics-aware dynamics absent in prior static methods. Code: https://github.com/AIGeeksGroup/Code2Worlds. Website: https://aigeeksgroup.github.io/Code2Worlds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。