让视频生成模型学会多镜头连贯性，一次训练即可生成长视频。
Long Context Tuning for Video Generation
- 扩展注意力机制覆盖整个场景，支持多镜头联合生成。
- 无需额外参数，实现一致且可交互的长视频生成。
- 适合需要连续叙事视频的创作者和研究者使用。
近期视频生成技术已能生成长达一分钟的单镜头逼真视频，但真实叙事视频需多镜头场景并保持视觉与动态一致性。本文提出长上下文微调（Long Context Tuning, LCT），通过扩展预训练单镜头视频扩散模型的上下文窗口，直接从数据中学习场景级一致性。该方法将全注意力机制从单镜头扩展至整个场景，引入交错3D位置编码与异步噪声策略，实现无需额外参数的联合与自回归镜头生成。经过双向注意力训练后，可进一步用上下文因果注意力微调，支持高效键值缓存的自回归生成。实验表明，经过LCT的单镜头模型能生成连贯的多镜头场景，并展现出组合生成与交互式镜头扩展等新能力，为更实用的视觉内容创作铺路。
原文摘要 · Abstract (English)
Recent advances in video generation can produce realistic, minute-long single-shot videos with scalable diffusion transformers. However, real-world narrative videos require multi-shot scenes with visual and dynamic consistency across shots. In this work, we introduce Long Context Tuning (LCT), a training paradigm that expands the context window of pre-trained single-shot video diffusion models to learn scene-level consistency directly from data. Our method expands full attention mechanisms from individual shots to encompass all shots within a scene, incorporating interleaved 3D position embedding and an asynchronous noise strategy, enabling both joint and auto-regressive shot generation without additional parameters. Models with bidirectional attention after LCT can further be fine-tuned with context-causal attention, facilitating auto-regressive generation with efficient KV-cache. Experiments demonstrate single-shot models after LCT can produce coherent multi-shot scenes and exhibit emerging capabilities, including compositional generation and interactive shot extension, paving the way for more practical visual content creation. See https://guoyww.github.io/projects/long-context-video/ for more details.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。