新基准测试交互世界模型长期稳定性,覆盖动作、视觉、物理和记忆四维度。
WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models

- 设计四维评估体系,精准捕捉传统方法忽略的失效问题。
- 10+模型测试显示无一在所有维度表现可靠,最优仅达中等水平。
- 适合研究真实场景部署的交互世界模型的开发者与评估者。
尽管交互式世界模型(IWMs)进展迅速,现有基准仅在轨迹层面评估动作跟随,忽略记忆与交互物理特性。我们提出WorldRoamBench,一个开放世界长时序稳定性基准,涵盖四个维度并各有创新:(i) 动作:逐帧动作度量,规避跨模型语义尺度差异,暴露轨迹级评估隐藏的失败;(ii) 视觉:基于分割的漂移度量,捕捉起始与终点对比遗漏的非单调中间序列崩溃;(iii) 物理:在力学、光学与3D一致性上进行可控性门控评估,评分在忠实动作执行下的合理性;(iv) 记忆:动作解耦协议,通过过渡局部化3D点云重建评估场景记忆,结合跟踪+视觉语言模型推理评估主体记忆。该基准包含600多个测试用例,覆盖自然、城市、室内场景,第一/第三人称视角,支持WASD键连续交互10-60秒。对10+开源/闭源模型的评估显示,无一模型在所有维度表现可靠,即使最优者也仅达到中等分数。在WorldRoamBench上的进步是迈向稳定、物理合理、记忆忠实且可部署于真实应用的IWMs的关键一步。
原文摘要 · Abstract (English)
Despite rapid progress in interactive world models (IWMs), existing benchmarks evaluate action following only at trajectory level and ignore memory and interaction physics. We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions, each with tailored innovations: (i) Action: per-frame action metric bypassing cross-model semantic scale disparity and exposing failures hidden by trajectory; (ii) Vision: segment-based drift metric capturing non-monotonic mid-sequence collapse missed by start-vs-end comparisons; (iii) Physics: controllability-gated evaluation over mechanics, optics, and 3D consistency, scoring plausibility under faithful action execution; (iv) Memory: action-decoupled protocol evaluating scene memory via transition-localized 3D point-cloud reconstruction and subject memory via tracking-plus-VLM reasoning. The benchmark comprises 600+ test cases across Nature, Urban, and Indoor scenes in first/third-person views with WASD 10-60s continuous interaction. Evaluating 10+ open/closed-source models reveals none reliably satisfies all dimensions; even the best achieves only moderate scores. Advances on WorldRoamBench are steps toward IWMs that are stable, physically grounded, memory-faithful, and deployable in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。