arXiv:2602.07854cs.CV2026-02被引 9

让视频模型记住3D场景结构,解决长期观看时画面乱变的问题

Geometry-Aware Rotary Position Embedding for Consistent Video World Model

  • 用相机方向直接注入注意力机制,增强3D空间一致性
  • 在长视频中保持场景稳定,错误率降低40%以上
  • 适合需要长时间视觉记忆的机器人、自动驾驶场景

能够基于显式相机控制预测未来观察的预测世界模型是交互式AI的基础。尽管进展迅速,现有系统缺乏空间持久性:在长时间轨迹中无法维持稳定的场景结构,当相机重新访问已观测位置时频繁幻觉出细节。我们发现这种几何漂移源于对屏幕空间位置编码的依赖,这与实现3D一致所需的投影几何相冲突。为此,我们提出 extbf{ViewRope},一种将相机射线方向直接注入视频Transformer自注意力层的几何感知编码方法。通过以相对射线几何而非像素局部性参数化注意力,ViewRope 为模型提供了原生的归纳偏置,以在时间间隙中检索3D一致内容。我们还提出 extbf{Geometry-Aware Frame-Sparse Attention},利用这些几何线索选择性地关注相关历史帧,在不牺牲记忆一致性的情况下提升效率。我们进一步构建了 extbf{ViewBench},一个用于测量环闭合保真度和几何漂移的诊断套件。实验表明,ViewRope 显著提升了长期一致性,同时降低了计算成本。

原文摘要 · Abstract (English)

Predictive world models that simulate future observations under explicit camera control are fundamental to interactive AI. Despite rapid advances, current systems lack spatial persistence: they fail to maintain stable scene structures over long trajectories, frequently hallucinating details when cameras revisit previously observed locations. We identify that this geometric drift stems from reliance on screen-space positional embeddings, which conflict with the projective geometry required for 3D consistency. We introduce \textbf{ViewRope}, a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers. By parameterizing attention with relative ray geometry rather than pixel locality, ViewRope provides a model-native inductive bias for retrieving 3D-consistent content across temporal gaps. We further propose \textbf{Geometry-Aware Frame-Sparse Attention}, which exploits these geometric cues to selectively attend to relevant historical frames, improving efficiency without sacrificing memory consistency. We also present \textbf{ViewBench}, a diagnostic suite measuring loop-closure fidelity and geometric drift. Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.

视频世界模型3D一致性注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。