提出FastInit，单次前向传播实现视频生成的高效噪声优化。
FastInit: Fast Noise Initialization for Temporally Consistent Video Generation
- 构建VNPNet，输入随机噪声与文本提示，一步生成优化噪声。
- 在多个文生视频模型上提升画面质量和帧间一致性。
- 无需迭代优化，推理效率显著提高，适合实际部署。
视频生成得益于扩散模型的发展取得了显著进展，但保持高时间一致性仍是挑战。近期的FreeInit发现训练-推理差距，并提出通过推理阶段迭代精炼初始噪声的方法。然而，迭代过程显著增加了计算开销。本文提出FastInit，一种快速噪声初始化方法，无需迭代精炼。FastInit学习一个视频噪声预测网络（VNPNet），以随机噪声和文本提示为输入，在单次前向传播中生成优化噪声，大幅提升了视频生成效率，同时保证帧间高一致性。为训练VNPNet，我们构建了一个大规模数据集，包含文本提示、随机噪声与对应优化噪声的配对。在多种文生视频模型上的大量实验表明，该方法持续提升生成视频的质量与时间一致性。FastInit不仅显著改进了视频生成性能，还提供了一种可直接应用于推理阶段的实用方案。代码与数据集将公开。
原文摘要 · Abstract (English)
Video generation has made significant strides with the development of diffusion models; however, achieving high temporal consistency remains a challenging task. Recently, FreeInit identified a training-inference gap and introduced a method to iteratively refine the initial noise during inference. However, iterative refinement significantly increases the computational cost associated with video generation. In this paper, we introduce FastInit, a fast noise initialization method that eliminates the need for iterative refinement. FastInit learns a Video Noise Prediction Network (VNPNet) that takes random noise and a text prompt as input, generating refined noise in a single forward pass. Therefore, FastInit greatly enhances the efficiency of video generation while achieving high temporal consistency across frames. To train the VNPNet, we create a large-scale dataset consisting of pairs of text prompts, random noise, and refined noise. Extensive experiments with various text-to-video models show that our method consistently improves the quality and temporal consistency of the generated videos. FastInit not only provides a substantial improvement in video generation but also offers a practical solution that can be applied directly during inference. The code and dataset will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。