arXiv:2601.05966cs.CVcs.AI2026-01被引 5

首个大规模自回归视频生成框架,高效生成高质量视频。

VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction

  • 分层预测下一帧与多尺度特征,解耦时空依赖。
  • 在UCF-101上FVD降至88.6,推理步数减少10倍以上。
  • 适合追求高效、长时一致性视频生成的研究者。

近期视频生成主要依赖扩散和流匹配模型,虽质量高但计算开销大且难以扩展。本文提出VideoAR,首个大规模视觉自回归(VAR)框架,结合多尺度下一帧预测与自回归建模。通过引入3D多尺度分词器,将帧内自回归建模与因果下一帧预测结合,有效编码时空动态。为提升长期一致性,提出多尺度时间位置编码(Multi-scale Temporal RoPE)、跨帧误差校正与随机帧掩码,共同缓解误差传播并稳定时序连贯性。采用多阶段预训练流程,逐步对齐不同分辨率与时长下的时空学习。实验表明,VideoAR在自回归模型中达到新基准:UCF-101上FVD从99.5降至88.6,推理步数减少超10倍;VBench得分达81.74,媲美参数量大一个数量级的扩散模型。结果证明,VideoAR缩小了自回归与扩散范式间的性能差距,为未来视频生成研究提供高效、可扩展、时序一致的新基础。

原文摘要 · Abstract (English)

Recent advances in video generation have been dominated by diffusion and flow-matching models, which produce high-quality results but remain computationally intensive and difficult to scale. In this work, we introduce VideoAR, the first large-scale Visual Autoregressive (VAR) framework for video generation that combines multi-scale next-frame prediction with autoregressive modeling. VideoAR disentangles spatial and temporal dependencies by integrating intra-frame VAR modeling with causal next-frame prediction, supported by a 3D multi-scale tokenizer that efficiently encodes spatio-temporal dynamics. To improve long-term consistency, we propose Multi-scale Temporal RoPE, Cross-Frame Error Correction, and Random Frame Mask, which collectively mitigate error propagation and stabilize temporal coherence. Our multi-stage pretraining pipeline progressively aligns spatial and temporal learning across increasing resolutions and durations. Empirically, VideoAR achieves new state-of-the-art results among autoregressive models, improving FVD on UCF-101 from 99.5 to 88.6 while reducing inference steps by over 10x, and reaching a VBench score of 81.74-competitive with diffusion-based models an order of magnitude larger. These results demonstrate that VideoAR narrows the performance gap between autoregressive and diffusion paradigms, offering a scalable, efficient, and temporally consistent foundation for future video generation research.

视频生成自回归多尺度高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。