用语义槽替代时间记忆，让长视频生成更连贯
SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation
- 将记忆从时间顺序转为物体语义槽，实现对象级持久化
- 在60秒交互叙事中达到81.61分质量分，动态一致性提升22.8%
- 适合需要长期对象一致性的视频生成研究者使用
流式视频生成模型通常依赖时间中心的记忆机制，将历史上下文组织为原始帧、片段或未聚类的标记。这种组织方式在实体离开画面或交互提示切换时，常导致身份漂移和语义不一致。为此，我们提出SlotMemory，一种面向流式视频扩散模型的对象中心键值记忆机制。该方法将Transformer的键值流形分解为离散可复用的语义槽，将记忆抽象从“事件何时发生”转向“表示的是什么”。通过这些槽作为路由地址索引并存储高保真键值标记，实现跨长时程的实体级持久性和提示感知检索。在使用Wan2.1-T2V-1.3B主干模型的60秒交互叙事任务上，SlotMemory取得81.61的最优质量分数，并相比最强现有基线实现22.8%的动态一致性相对提升。结果表明，结构化语义表征而非原始时间容量，是持久长视频合成的核心基础。
原文摘要 · Abstract (English)
Streaming video generation models typically rely on temporal-centric memory, which organizes historical context as raw frames, chunk segments, or unclustered tokens. This organization frequently leads to identity drift and semantic inconsistency when entities exit the frame or during interactive prompt transitions. To address these limitations, we propose SlotMemory, an object-centric Key-Value memory mechanism for streaming video diffusion. Our approach shifts the memory abstraction from "when" an event occurred to "what" is being represented by decomposing the transformer's key-value manifold into discrete, reusable semantic slots. By utilizing these slots as routing addresses to index and store high-fidelity key-value tokens, we enable entity-level persistence and prompt-aware retrieval across long horizons. Evaluated on 60-second interactive narratives using the Wan2.1-T2V-1.3B backbone, SlotMemory achieves a state-of-the-art quality score of 81.61 and a 22.8 percent relative improvement in dynamic consistency over the strongest existing streaming baseline. Our results demonstrate that structured semantic representation, rather than raw temporal capacity, is the essential primitive for persistent long-form video synthesis. Our codes and checkpoints are available at https://tj12323.github.io/SlotMemory/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。