无需训练即可生成超长视频,保持帧间一致性。
Enhancing Train-Free Infinite-Frame Generation for Consistent Long Videos

- 分两阶段对齐训练与推理,减少模型输入噪声。
- 结合自反思与远距帧引导,提升长时间生成一致性。
- 适合需要无限长度视频生成的科研与应用开发。
无显著计算开销的免训练长视频生成旨在使基础视频生成模型能够生成更长的视频序列。帧级自回归框架(如 FIFO-diffusion)可实现无限长视频生成且内存消耗恒定。然而,训练与推理之间的不匹配以及长期一致性难以维持的问题,限制了基础模型的有效利用。为此,我们提出一种新型无限帧长视频生成方法 MIGA。首先,设计了一种有效的两阶段对齐机制,通过减少输入模型的过量噪声跨度来缓解训练-推理差异。随后,引入创新的双一致性增强机制:自反思策略修正早期高噪声帧,远距帧引导策略利用后期低噪声、覆盖范围广的帧进行生成引导,协同提升时间一致性。在 VBench 与 NarrLV 多项实验中,MIGA 表现达到当前最优水平。
原文摘要 · Abstract (English)
Without incurring significant computational overhead, train-free long video generation aims to enable foundation video generation models to produce longer videos. Frame-level autoregressive frameworks, e.g., FIFO-diffusion, offer the advantage of generating infinitely long videos with constant memory consumption. However, the mismatch between training and inference, coupled with the challenge of maintaining long-term consistency, limits the effective utilization of foundation models. To mitigate these concerns, we propose \textbf{MIGA}, a novel infinite-frame long video generation method. Firstly, we propose an effective two-stage alignment mechanism that mitigates the training-inference gap by reducing the excessive noise span fed to the model. We then introduce an innovative dual consistency enhancement mechanism, where the self-reflection approach corrects early high-noise frames and the long-range frame guidance approach leverages later low-noise frames with broad coverage to steer generation, jointly improving temporal consistency. Extensive experiments on VBench and NarrLV demonstrate the state-of-the-art performance of MIGA. Our project page is available at https://xiaokunfeng.github.io/miga_homepage/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。