让自回归视频生成突破时间限制,实现无限长、可控、电影级切换的视频生成。
Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout
- 用相对时序编码替代固定位置编码,突破3D-RoPE的时间上限
- 仅保留两帧缓存,实现即时动作控制响应
- 支持单次生成内多剪辑场景切换,适合影视创作
当前自回归视频扩散模型受限于三大瓶颈:(i) 基础模型3D旋转位置编码(3D-RoPE)带来的有限时间范围;(ii) 长序列生成中精细动作控制的提示响应缓慢;(iii) 单次生成流内无法实现非连续电影式转场。本文提出∞-RoPE,一个统一的推理时框架,通过三个相互关联组件解决上述问题:块相对旋转位置编码(Block-Relativistic RoPE)将时间编码重构为移动局部参考系,使新生成潜空间块相对于模型最大帧数进行旋转,而早期块则反向旋转以保持相对时序几何结构。该相对性设计消除了固定时间位置,实现远超基础位置编码限制的连续视频生成。为实现无重编码的精细动作控制,KV Flush通过仅保留全局汇点和最后一帧潜变量来刷新键值缓存,确保即时提示响应。最后,RoPE Cut引入受控的时间位置编码断点,支持单个连续生成流内的多剪辑场景转换。三者协同使∞-RoPE成为无需训练的无限时长、可控制、电影级视频扩散基础。大量实验表明,∞-RoPE在整体VBench评分上持续优于此前自回归模型。
原文摘要 · Abstract (English)
Current autoregressive video diffusion models are constrained by three core bottlenecks: (i) the finite temporal horizon imposed by the base model's 3D Rotary Positional Embedding (3D-RoPE), (ii) slow prompt responsiveness in maintaining fine-grained action control during long-form rollouts, and (iii) the inability to realize discontinuous cinematic transitions within a single generation stream. We introduce $\infty$-RoPE, a unified inference-time framework that addresses all three limitations through three interconnected components: Block-Relativistic RoPE, KV Flush, and RoPE Cut. Block-Relativistic RoPE reformulates temporal encoding as a moving local reference frame, where each newly generated latent block is rotated relative to the base model's maximum frame horizon while earlier blocks are rotated backward to preserve relative temporal geometry. This relativistic formulation eliminates fixed temporal positions, enabling continuous video generation far beyond the base positional limits. To obtain fine-grained action control without re-encoding, KV Flush renews the KV cache by retaining only two latent frames, the global sink and the last generated latent frame, thereby ensuring immediate prompt responsiveness. Finally, RoPE Cut introduces controlled discontinuities in temporal RoPE coordinates, enabling multi-cut scene transitions within a single continuous rollout. Together, these components establish $\infty$-RoPE as a training-free foundation for infinite-horizon, controllable, and cinematic video diffusion. Comprehensive experiments show that $\infty$-RoPE consistently surpasses previous autoregressive models in overall VBench scores.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。