arXiv:2607.01766cs.AI2026-07被引 1

用多智能体生成可编辑的动态3D场景,让文字描述自动生成会动的4D世界。

SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation

论文配图:SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
图 1 · 摘自论文原文
  • 分阶段多智能体协作,规划-编码-审查流程确保场景一致性
  • 在4DBuildBench上生成的动态场景物理正确率超基线18.7%以上
  • 适合做视频生成与具身AI训练数据的开发者和研究者

大型语言模型智能体正被用于以程序化方式将自然语言转为3D场景,但现有系统多聚焦静态输出。仅凭文本生成包含液体流动、粒子发射、刚体下落、关节机构运动等动态元素的4D场景仍属空白,而这类内容对可编辑性及视频生成与具身人工智能的物理驱动训练极具价值。两大挑战使动态场景生成区别于静态任务:一是需协同控制空间布局、多个物理求解器、时间序列、相机与光照;二是从渲染视频验证运动正确性远比判断单张图像困难。本文提出SimWorlds:一个基于多智能体框架的动态4D场景生成系统,融合Blender专属程序知识,采用规划-编码-审查工作流,按固定顺序执行构建阶段;通过确定性验证器强制执行分层场景协议,并配备运行时状态检测工具套件,捕捉渲染图像无法揭示的机制故障。我们还引入4DBuildBench基准,用于评估从文本提示生成的动态3D场景的视觉保真度与物理一致性。实验表明,SimWorlds在动态场景生成方面优于现有基线。

原文摘要 · Abstract (English)

LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI. Two challenges set the dynamic case apart from static text-to-scene work: an agent must jointly coordinate spatial layout, multiple physics solvers, temporal sequencing, camera, and lighting in a single coherent scene, and verifying motion correctness from rendered video is fundamentally harder than judging a single image. We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text, with Blender-specific procedural knowledge, a planner-coder-reviewer workflow driving a fixed ordered sequence of construction stages, a layered scene protocol enforced by a deterministic verifier, and a runtime-state inspection tool suite that catches mechanism failures the rendered image cannot reveal. We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes generated from text prompts. Experiments show that SimWorlds outperforms prior dynamic Blender generation baselines.

3D生成多智能体动态场景物理模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。