让视频世界模型记住场景几何结构，提升长期生成一致性。
Geometry-Aware Implicit Memory for Video World Models
- 用轻量Transformer压缩历史，显式编码跨视角三维结构
- 在MIND数据集上长时序几何与视觉一致性优于基线方法
- 适合需要稳定长期视频生成的场景建模任务
视频世界模型旨在模拟可控制的视觉环境，但长期推理依赖于观测离开原始上下文窗口后的记忆能力。显式记忆存储帧或在线3D重建，易受启发式检索错误、冗余外观存储或重建伪影影响；隐式记忆将历史压缩为紧凑状态，但现有设计未显式约束对跨视角场景几何的编码。本文提出GIM-World，一种面向视频世界模型的几何感知隐式记忆框架：轻量级Transformer编码器将变长历史压缩为固定大小的记忆令牌；一个可相机查询的几何头，在训练中从冻结的基础模型中提炼3D场景结构并注入记忆；信息引导的剪枝规则确保随历史增长编码成本可控。推理时几何教师被移除，仅保留轻量记忆模块。在MIND数据集上的实验表明，GIM-World在长时序下比显式与隐式记忆基线更好地保持几何与视觉一致性。
原文摘要 · Abstract (English)
Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit memories retain frames or online 3D reconstructions, which can suffer from heuristic retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit memory framework for video world models. A lightweight transformer encoder compresses variable-length history into fixed-size memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training, and an information-guided pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight memory module. Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。