动态检索历史帧，让长视频生成更连贯且高效。
MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives
- 根据当前文本提示动态更新记忆库，选最相关的历史帧。
- 生成时只激活相关记忆令牌，效率仅降7.9%。
- 适合需要长视频连贯性的生成任务，兼容主流模型。
流式视频生成的核心挑战在于长上下文中的内容一致性，这对记忆设计提出高要求。现有方法通常用预设策略压缩历史帧以维护记忆，但不同生成片段应参考不同的历史线索，固定策略难以满足。本文提出MemFlow，针对即将生成的视频块，先根据其文本提示动态检索最相关的历史帧来更新记忆库，从而在新事件或场景切换时仍保持叙事连贯性。生成过程中，仅在注意力层中激活记忆库中最相关的令牌，有效保障生成效率。该方法在实现卓越长上下文一致性的同时，计算开销几乎可忽略（相比无记忆基线仅降低7.9%速度），且兼容任何使用键值缓存（KV cache）的流式视频生成模型。
原文摘要 · Abstract (English)
The core challenge for streaming video generation is maintaining the content consistency in long context, which poses high requirement for the memory design. Most existing solutions maintain the memory by compressing historical frames with predefined strategies. However, different to-generate video chunks should refer to different historical cues, which is hard to satisfy with fixed strategies. In this work, we propose MemFlow to address this problem. Specifically, before generating the coming chunk, we dynamically update the memory bank by retrieving the most relevant historical frames with the text prompt of this chunk. This design enables narrative coherence even if new event happens or scenario switches in future frames. In addition, during generation, we only activate the most relevant tokens in the memory bank for each query in the attention layers, which effectively guarantees the generation efficiency. In this way, MemFlow achieves outstanding long-context consistency with negligible computation burden (7.9% speed reduction compared with the memory-free baseline) and keeps the compatibility with any streaming video generation model with KV cache.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。