arXiv:2511.18886cs.CV2025-11被引 1

解决交互式视频世界模型长时稳定性的运动漂移与误差累积问题

MagicWorld: Towards Long-Horizon Stability for Interactive Video World Exploration

  • 引入流引导运动保持约束,提升动态主体运动真实性
  • 采用历史缓存检索与多示例聚合蒸馏,显著减少长期交互误差
  • 适用于需要高稳定性长时交互的虚拟场景生成任务

近期交互式视频世界模型可基于用户指令生成场景演化,但仍存在两大缺陷:在复杂环境中,动态主体易出现运动漂移,无法遵循真实运动模式;在长时交互中,自回归生成导致误差累积,引发结构与语义不一致。本文提出MagicWorld,基于自回归框架构建交互式视频世界模型。为缓解运动漂移,引入流引导运动保持约束,增强动态主体运动真实性与交互稳定性。为降低长时误差累积,设计两种互补策略:历史缓存检索机制通过回溯过往生成状态强化历史一致性;增强交互训练策略采用多示例聚合蒸馏与双奖励加权,提升长期建模能力。此外,构建RealWM120K数据集,包含12万条真实城市漫步视频及多模态标注,支持动态感知与长时建模。实验表明,MagicWorld显著提升运动真实性和长时交互稳定性。

原文摘要 · Abstract (English)

Recent interactive video world model methods generate scene evolution conditioned on user instructions. Although they achieve impressive results, two key limitations remain. First, they exhibit motion drift in complex environments with multiple interacting subjects, where dynamic subjects fail to follow realistic motion patterns during scene evolution. Second, they suffer from error accumulation in long-horizon interactions, where autoregressive generation gradually drifts from earlier scene states and causes structural and semantic inconsistencies. In this paper, we propose MagicWorld, an interactive video world model built upon an autoregressive framework. To address motion drift, we incorporate a flow-guided motion preservation constraint that mitigates motion degradation in dynamic subjects, encouraging realistic motion patterns and stable interactions during scene evolution. To mitigate error accumulation in long-horizon interactions, we design two complementary strategies, including a history cache retrieval strategy and an enhanced interactive training strategy. The former reinforces historical scene states by retrieving past generations during interaction, while the latter adopts multi-shot aggregated distillation with dual-reward weighting for interactive training, enhancing long-term stability and reducing error accumulation. In addition, we construct RealWM120K, a real-world dataset with diverse city-walk videos and multimodal annotations to support dynamic perception and long-horizon world modeling. Experimental results demonstrate that MagicWorld improves motion realism and alleviates error accumulation during long-horizon interactions.

视频世界模型长时稳定交互生成运动保持

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。