通过流形对齐实现快速低内存视频续写，效果超越现有方法。
FlowC2S: Flowing from Current to Succeeding Frames for Fast and Memory-Efficient Video Continuation
- 直接从当前帧流到后续帧，输入维度减半，提升效率。
- 使用相邻视频块作为最优耦合近似，生成更平滑的运动轨迹。
- 适合需要低延迟、低显存的实时视频生成场景。
本文提出一种新型视频续写方法 FlowC2S，通过微调预训练的文本到视频流模型，学习当前与后续视频块之间的向量场。核心设计包括：利用时间相邻视频块作为真实最优耦合的实用代理，使运动流更平直；引入目标反演机制，将目标块的反向隐变量注入输入表示，增强对应关系并提升视觉保真度。相比传统先融合当前帧与噪声再生成的方法，本方法直接从当前帧流向后续帧，使模型输入维度降低两倍。基于 LTXV 与 Wan 模型微调，该方法在 FID 与 FVD 评价指标上均超越现有技术，仅需五次神经函数评估即可达成最佳性能。
原文摘要 · Abstract (English)
This paper introduces a novel methodology for generating fast and memory-efficient video continuations. Our method, dubbed FlowC2S, fine-tunes a pre-trained text-to-video flow model to learn a vector field between the current and succeeding video chunks. Two design choices are key. First, we introduce inherent optimal couplings, utilizing temporally adjacent video chunks during training as a practical proxy for true optimal couplings, resulting in straighter flows. Second, we incorporate target inversion, injecting the inverted latent of the target chunk into the input representation to strengthen correspondences and improve visual fidelity. By flowing directly from current to succeeding frames, instead of the common combination of current frames with noise to generate a video continuation, we reduce the dimensionality of the model input by a factor of two. The proposed method, fine-tuned from LTXV and Wan, surpasses the state-of-the-art scores across quantitative evaluations with FID and FVD, with as few as five neural function evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。