arXiv:2512.04040cs.CV2025-12被引 74

RELIC实现实时长时交互视频生成,支持稳定空间记忆与精准用户控制。

RELIC: Interactive Video World Model with Long-Horizon Memory

  • 用压缩潜变量+相机位姿编码长时记忆,兼顾实时性与一致性。
  • 14B参数模型在16帧/秒下实现5秒以上流畅生成,优于以往方法。
  • 适合需要长期一致性的虚拟世界交互场景,如游戏、元宇宙开发。

真正的交互式世界模型需要三个关键要素:实时长时流式处理、一致的空间记忆和精确的用户控制。然而,现有方法大多仅解决其中一项,同时实现三者极具挑战——例如,长时记忆机制常损害实时性能。本文提出统一框架RELIC,可基于单张图像与文本描述,实现实时、带记忆的任意场景长时探索。基于最新的自回归视频扩散蒸馏技术,模型通过在键值缓存中编码相对动作与绝对相机位姿,以高度压缩的历史潜变量表示长时记忆。该紧凑且相机感知的记忆结构支持隐式三维一致性内容检索,并以极低计算开销维持长期连贯性。同时,我们微调一个双向教师视频模型,使其生成序列超出原始5秒训练范围,并通过一种新的内存高效自强制范式,将教师模型的完整上下文蒸馏至因果学生生成器,实现长时间教师与学生自回溯的完整蒸馏。该模型为140亿参数,基于精心筛选的Unreal Engine渲染数据集训练,可在16帧/秒下实现实时生成,相较先前工作展现出更准确的动作跟随、更稳定的长时流式输出以及更鲁棒的空间记忆检索能力。这些能力使RELIC成为下一代交互式世界建模的坚实基础。

原文摘要 · Abstract (English)

A truly interactive world model requires three key ingredients: real-time long-horizon streaming, consistent spatial memory, and precise user control. However, most existing approaches address only one of these aspects in isolation, as achieving all three simultaneously is highly challenging-for example, long-term memory mechanisms often degrade real-time performance. In this work, we present RELIC, a unified framework that tackles these three challenges altogether. Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time. Built upon recent autoregressive video-diffusion distillation techniques, our model represents long-horizon memory using highly compressed historical latent tokens encoded with both relative actions and absolute camera poses within the KV cache. This compact, camera-aware memory structure supports implicit 3D-consistent content retrieval and enforces long-term coherence with minimal computational overhead. In parallel, we fine-tune a bidirectional teacher video model to generate sequences beyond its original 5-second training horizon, and transform it into a causal student generator using a new memory-efficient self-forcing paradigm that enables full-context distillation over long-duration teacher as well as long student self-rollouts. Implemented as a 14B-parameter model and trained on a curated Unreal Engine-rendered dataset, RELIC achieves real-time generation at 16 FPS while demonstrating more accurate action following, more stable long-horizon streaming, and more robust spatial-memory retrieval compared with prior work. These capabilities establish RELIC as a strong foundation for the next generation of interactive world modeling.

世界模型视频生成长时记忆交互生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。