arXiv:2603.07145cs.CV2026-03被引 16

让视频世界模型在视线外也能持续演化,实现真正的动态世界模拟。

LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models

  • 构建全局持久状态,未被观察的物体仍持续变化。
  • 引入监控机制自动推进未见物体状态,重访时保持一致。
  • 提出新基准LiveBench,评估长期场景一致性。

近期生成式视频世界模型旨在模拟视觉环境的演变,使观察者可通过摄像头控制交互探索场景。然而,它们隐含假设世界仅在观察视野内演化:一旦物体离开视野,其状态便被冻结在记忆中,再次返回时往往无法反映其间应发生的事件。本文将此被忽视的局限性定义为“视线外动态”问题,阻碍了视频世界模型对连续演化的世界进行建模。为此,我们提出LiveWorld框架,扩展视频世界模型以支持持久的世界演化。不同于将世界视为静态观测记忆,LiveWorld建模一个由静态3D背景和持续演化的动态实体组成的全局持久状态。为维持未被观察的动态,LiveWorld引入基于监控的机制,自主模拟活跃实体的时间演化,并在重新访问时同步其演化状态,确保空间渲染的一致性。为进一步评估,我们还提出了LiveBench,一个专门用于维护视线外动态的任务基准。大量实验表明,LiveWorld实现了持久事件演化与长期场景一致性,弥合了现有基于2D观测记忆与真正4D动态世界模拟之间的差距。基线和基准代码将公开于https://zichengduan.github.io/LiveWorld/index.html。

原文摘要 · Abstract (English)

Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.

视频生成世界模型动态演化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。