无需训练即可扩展自回归视频生成时长,解决长视频生成中的误差累积问题。
Train Short, Inference Long: Training-free Horizon Extension for Autoregressive Video Generation
- 通过频域感知的位置编码调制,动态插补低频成分并外推高频信息。
- 在VBench上实现6倍时长(30秒)外推性能超越现有模型,12倍时长(60秒)媲美微调模型。
- 可直接嵌入现有推理流程,适用于如LongLive等模型的4分钟级视频生成。
自回归视频扩散模型已成为生成长视频的可扩展范式,但常因严重外推失败导致时序退化:误差快速累积使超出训练时长的生成质量显著下降。本文发现该问题主要源于三维位置编码的谱偏差以及噪声采样中缺乏动态先验。为此提出FLEX(Frequency-aware Length EXtension),一种无需训练的推理时框架,弥合短时训练与长时推理间的差距。FLEX引入频域感知的RoPE调制,自适应插补未充分训练的低频分量,同时外推高频分量以保持多尺度时序判别力;结合反相噪声采样(ANS)注入高频动态先验,并采用仅推理注意力池锚定全局结构。在VBench上的大量实验表明,FLEX在6倍外推(30秒)下显著优于现有最先进模型,在12倍尺度(60秒)上达到长视频微调基线水平。作为即插即用的增强模块,FLEX可无缝集成至现有推理管线,有效推动LongLive等模型实现4分钟级稳定动态视频生成。
原文摘要 · Abstract (English)
Autoregressive video diffusion models have emerged as a scalable paradigm for long video generation. However, they often suffer from severe extrapolation failure, where rapid error accumulation leads to significant temporal degradation when extending beyond training horizons. We identify that this failure primarily stems from the spectral bias of 3D positional embeddings and the lack of dynamic priors in noise sampling. To address these issues, we propose FLEX (Frequency-aware Length EXtension), a training-free inference-time framework that bridges the gap between short-term training and long-term inference. FLEX introduces Frequency-aware RoPE Modulation to adaptively interpolate under-trained low-frequency components while extrapolating high-frequency ones to preserve multi-scale temporal discriminability. This is integrated with Antiphase Noise Sampling (ANS) to inject high-frequency dynamic priors and Inference-only Attention Sink to anchor global structure. Extensive evaluations on VBench demonstrate that FLEX significantly outperforms state-of-the-art models at 6x extrapolation (30s duration) and matches the performance of long-video fine-tuned baselines at 12x scale (60s duration). As a plug-and-play augmentation, FLEX seamlessly integrates into existing inference pipelines for horizon extension. It effectively pushes the generation limits of models such as LongLive, supporting consistent and dynamic video synthesis at a 4-minute scale. Project page is available at https://ga-lee.github.io/FLEX_demo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。