用智能体玩家测试世界模型的长期交互能力,发现现有模型在空间一致性上仍不可靠。
PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

- 用多模态智能体主动探索环境,按目标评估模型表现
- 9个顶尖模型在长期任务中普遍存在空间一致性差的问题
- 适合关注世界模型真实性和长期推理能力的研究者
视频世界模型能根据当前观测和用户动作预测未来状态。近期系统在长序列上展现出出色的视频一致性和动作可控性,但公平比较这些交互模型仍具挑战。实践中,人类玩家通常通过与环境互动完成长期目标来评估模型,例如绕行360度检查环境一致性,或走入水中观察是否生成真实的水波纹。实现相同目标所需的动作序列在不同模型间差异显著,使得固定动作条件下的评估不适用于跨模型比较。为此,我们采用多模态智能体玩家与世界模型交互,达成指定的长期目标。基于此范式,我们构建PlayWorld基准,包含171个场景及对应目标。为全面评估性能,我们从几何一致性、交互保真度、视野外演化和洞察演化四个核心维度进行测评,并补充视频质量与可控性基础指标。对九个前沿世界模型的实验表明,当前模型在长期交互目标下仍不可靠,尤其在维持空间一致性和持续状态演化方面存在明显缺陷。代码与数据已开源。
原文摘要 · Abstract (English)
Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives. Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。