arXiv:2608.13492cs.AI2026-08被引 1

让视频生成更连贯,通过动态条件匹配提升长时序建模能力

AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)

论文配图:AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)
图 1 · 摘自论文原文
  • 用动态点云缓存替代静态图像,实现运动感知的条件输入
  • 条件信号与生成内容共享同一潜空间,时间结构对齐提升一致性
  • 适合关注长视频生成、视觉一致性的研究者和开发者

本报告介绍 AlayaWorld 的改进版本。尽管骨干架构、分块自回归生成方案和训练数据与前一版保持一致,我们大幅重构了条件信号的表示与融合方式。新设计遵循一个核心原则:条件信号在潜在表示和时间结构上应尽可能与生成内容匹配。为此,我们进行两项关键改动:首先,将原有的基于深度扭曲的空间记忆替换为流式 3D 点缓存渲染器;其次,重新设计条件处理流程,使视觉条件在相同的因果变分自编码器(causal-VAE)潜空间中编码,且时间统计特性与生成视频一致。具体引入六项改进:(1) 用运动感知的潜空间条件替代静态帧输入;(2) 将重渲染的空间记忆以连续序列形式因果编码;(3) 在像素空间对齐时间记忆窗口;(4) 采用硬内存丢弃策略,移除记忆令牌而非置零;(5) 统一训练与推理阶段的 VAE 编码解码协议;(6) 移除相机 AdaLN 分支,视角控制完全通过重渲染的空间条件实现。

原文摘要 · Abstract (English)

This report presents an improved version of AlayaWorld. While the backbone architecture, chunk-wise autoregressive generation scheme, and training data remain unchanged from the previous release, we substantially revise how conditioning signals are represented and integrated into the model. The new design is guided by a simple principle: conditioning signals should match the generated content as closely as possible in both latent representation and temporal structure. To this end, we make two major changes. First, we replace the previous depth-warping-based spatial memory with a streaming 3D point-cache renderer. Second, we redesign the conditioning pipeline so that visual conditions are encoded in the same causal-VAE latent space, with temporal statistics consistent with those of the generated video. Concretely, the new version introduces six modifications: (1) replacing static-frame image conditioning with motion-aware latent conditioning; (2) causally encoding re-rendered spatial memory as a continuous sequence; (3) aligning the temporal-memory window in pixel space; (4) adopting hard memory dropout that removes memory tokens rather than zeroing them; (5) unifying the VAE encoding and decoding protocol across training and inference; and (6) removing the camera AdaLN branch, such that viewpoint control is provided entirely through the re-rendered spatial condition.

世界模型长时视频条件生成3D渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。