评测大模型生成可交互3D世界的能力,聚焦物理与状态一致性。
WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis

- 构建包含2026个任务的基准,覆盖仿真、渲染与应用场景。
- 最佳模型仅达27.8%验证覆盖率,主要失败于状态变化和交互链断裂。
- 提出状态探针协议,可检测隐藏运行时契约,适配自动化测试者。
大型语言模型正被要求从自然语言生成可执行的交互式三维世界。以Three.js为基础的浏览器原生3D成为新前沿:生成程序需整合资源、遵守空间与物理约束,并保持用户界面与内部运行状态同步。现有网页生成评测多仅观察像素或DOM节点,而Three.js世界的动态行为隐藏在<canvas>内。本文提出WorldCoder-Bench,一个用于自主、物理具身3D世界合成的基准,包含2,026个专家标注的任务,覆盖仿真、渲染与应用场景,支持可选.glb资产与隐藏行为契约。我们进一步提出StateProbe,一种基于执行的探测协议,在沙箱浏览器中验证运行时状态与转换的隐藏契约。除验证覆盖率外,引入返回自动化率与时间效率倍数衡量修正后的成本与时间节省。在九个前沿模型中,最优系统在WorldCoder-Core上仅达27.8%验证覆盖率,在WorldCoder-Robust上为19.9%,失败主因是状态模式漂移与交互链断裂,而非缺少场景元素。实用指标显示,低成本或快速模型在简单领域仍具显著价值。代码与数据见https://anonymous.4open.science/r/WorldCoder-Bench/
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly asked not only to write static interfaces, but to construct executable interactive worlds from natural language. Browser-native 3D, commonly built with Three.js, is a natural next frontier: generated programs must integrate assets, obey spatial and physical constraints, and keep user-facing controls synchronized with hidden runtime state. Existing web-generation benchmarks and evaluators, however, largely observe only pixels or DOM nodes, while the mechanics of a Three.js world unfold inside an opaque <canvas>. We introduce WorldCoder-Bench, a benchmark for autonomous, physically grounded 3D world synthesis. WorldCoder-Bench contains 2,026 expert-curated tasks across Simulation, Rendering, and Application scenarios, with optional .glb assets and hidden behavioral contracts. We further propose StateProbe, an execution-based protocol that probes generated programs in a sandboxed browser and verifies hidden, mutation-hardened contracts over runtime states and transitions. Beyond verification coverage, we report Return on Automation and Time Efficiency Multiplier to measure correctness-adjusted cost and time savings. Across nine frontier models, the best system reaches only 27.8% verification coverage on WorldCoder-Core and 19.9% on WorldCoder-Robust, with failures dominated by state-schema drift and broken interaction chains rather than missing scene elements. Utility metrics further show that cheap or fast models can still provide substantial value on easier domains. WorldCoder-Bench is available at https://anonymous.4open.science/r/WorldCoder-Bench/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。