用稀疏路由机制实现分钟级长视频生成的稳定记忆。
Mixture of Contexts for Long Video Generation
- 动态选择关键片段与必选锚点，实现高效长程记忆检索。
- 在分钟级视频生成中保持角色、动作与场景一致性。
- 适合需要长时序一致性的视频生成研究者使用。
长视频生成本质上是长上下文记忆问题：模型需在长时间跨度内保留并检索关键事件，避免信息坍缩或漂移。然而，将扩散变压器扩展至长视频生成受限于自注意力的二次计算开销，导致内存与计算不可行，难以优化长序列。本文将长上下文视频生成重构为内部信息检索任务，提出可学习的稀疏注意力路由模块——上下文混合（Mixture of Contexts, MoC），作为有效的长期记忆检索引擎。在MoC中，每个查询动态选择少数信息丰富的片段及强制锚点（如字幕、局部窗口）进行关注，因果路由机制防止循环闭合。随着数据规模扩大和路由逐渐稀疏，模型将计算资源分配给显著历史内容，从而在数分钟的视频中保持身份、动作与场景的一致性。效率作为检索的副产品出现（近线性扩展），支持实际训练与生成，并在分钟级尺度上涌现出记忆与一致性。
原文摘要 · Abstract (English)
Long video generation is fundamentally a long context memory problem: models must retain and retrieve salient events across a long range without collapsing or drifting. However, scaling diffusion transformers to generate long-context videos is fundamentally limited by the quadratic cost of self-attention, which makes memory and computation intractable and difficult to optimize for long sequences. We recast long-context video generation as an internal information retrieval task and propose a simple, learnable sparse attention routing module, Mixture of Contexts (MoC), as an effective long-term memory retrieval engine. In MoC, each query dynamically selects a few informative chunks plus mandatory anchors (caption, local windows) to attend to, with causal routing that prevents loop closures. As we scale the data and gradually sparsify the routing, the model allocates compute to salient history, preserving identities, actions, and scenes over minutes of content. Efficiency follows as a byproduct of retrieval (near-linear scaling), which enables practical training and synthesis, and the emergence of memory and consistency at the scale of minutes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。