提出可动态演化的记忆机制,实现实时无限长视频生成。
Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation

- 用可学习的查询机制替代固定记忆策略,动态压缩历史帧
- 支持任意压缩比,计算开销不随视频长度增加
- 首次实现超24小时(超130万帧)实时滚动生成
我们提出 Echo-Infinity,一种自回归框架,实现实时无限视频生成。该框架采用可学习的演化记忆,在恒定计算成本下动态过滤、抽象和压缩任意长度的历史。现有方法多依赖预设的键值缓存调度、固定比例压缩或推理时的位置编码调整,这些设计不可避免地丢失历史信息并放大累积误差。受人类记忆巩固启发,Echo-Infinity 将人工记忆管理替换为可学习的记忆查询,当旧帧被移出局部窗口时,通过注意力和门控机制更新查询。这些查询与视频扩散变换器(DiTs)端到端联合优化,形成支持任意压缩比的演化记忆,且计算开销与视频长度无关。它们还作为通用生成先验,即使仅使用优化后的初始状态也能提升生成质量。我们进一步提出统一相对位置编码方案,将关键帧锚定在 id 0,新帧 id 最大不超过 DiTs 预训练的最大时间位置编码值,摆脱有限位置编码约束,弥合训练-推理外推差距。在长/短视频生成中均达到最先进性能,并首次展示约24小时(>130万帧)的实时滚动生成,表明向无限视频生成迈进的可行性路径。
原文摘要 · Abstract (English)
We present Echo Infinity, an autoregressive (AR) framework towards real-time infinite video generation that employs a learnable evolving memory to dynamically filter, abstract, and compress any-length history at constant cost. Existing methods mainly curate memory with predefined KV-cache schedules, fixed-ratio heuristic compression, or inference-time RoPE adaptation. These designs inevitably lose historical information and amplify compounding errors due to their limited cache window and ignorance of autoregressive generation noise. Inspired by human memory consolidation, Echo-Infinity replaces handcrafted memory curation with learnable Memory Query, which are updated by attention and a gating mechanism when past frames are evicted from the local window. The queries are optimized end-to-end with the video diffusion transformers (DiTs), forming an evolving memory that supports arbitrary compression ratios with constant computation independent of video length. They also act as a generalizable generation prior, improving quality even when only the optimized initial state is used. We further introduce Unified Relative RoPE Recipe, which anchors the sink frames to start from id 0 and lets the newest frame id grow at most to the DiTs' pretrained maximum temporal RoPE id throughout training and inference, freeing the model from the finite RoPE constraint and closing the train-test RoPE extrapolation gap. In long and short video generation, Echo-Infinity achieves state-of-the-art performance, and, to our knowledge, demonstrates promising 24-hour (>1.3 M frames) real-time rollouts for the first time, suggesting a practical path toward infinite video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。