arXiv:2511.12099cs.CV2025-11

提出自适应视频起始标记,提升长视频生成一致性与动态质量

Adaptive Begin-of-Video Tokens for Autoregressive Video Diffusion Models

  • 用可学习的起始标记动态吸收已生成帧,保持全局连贯性
  • 解耦采样轨迹与注意力窗口,改善局部运动细节与图像质量
  • 适合需要高连贯性和流畅动作的长视频生成场景

基于扩散模型的视频生成近期在短视频领域取得显著进展。为扩展至长视频生成,多数视频扩散模型采用自回归方式,即根据先前帧生成后续帧。现有方法主要有两种:块式扩展和流式去噪。前者直接拼接干净帧作为条件,存在去噪延迟与误差累积问题;后者维持噪声水平单调递增的去噪序列,每轮生成一帧并添加纯噪声,支持实时采样,但易导致一致性脆弱和运动动态不佳。本文提出自适应视频起始标记(ada-BOV),这是一种可学习的嵌入向量,通过类似自适应层归一化的调节机制,动态吸收已去噪帧信息,既保持全局一致性,又支持灵活条件输入。为进一步提升局部动态质量,我们设计了流式去噪优化策略,解耦采样轨迹长度与注意力窗口大小约束,增强局部引导能力。同时引入扰动增强训练噪声调度,平衡收敛速度与模型鲁棒性。大量实验表明,本方法在多项指标上均达到优异的定性与定量表现。

原文摘要 · Abstract (English)

Recent advancements in diffusion-based video generation have produced impressive and high-fidelity short videos. To extend these successes to generate coherent long videos, most video diffusion models (VDMs) generate videos in an autoregressive manner, i.e., generating subsequent frames conditioned on previous ones. There are generally two primary paradigms: chunk-based extension and stream denoising. The former directly concatenates previous clean frames as conditioning, suffering from denoising latency and error accumulation. The latter maintains the denoising sequence with monotonically increasing noise levels. In each denoising iteration, one clean frame is produced while a new pure noise is simultaneously appended, enabling live-stream sampling. However, it struggles with fragile consistency and poor motion dynamics. In this paper, we propose Adaptive Begin-of-Video Tokens (ada-BOV) for autoregressive VDMs. The BOV tokens are special learnable embeddings on VDMs. They adaptively absorb denoised preceding frames via an adaptive-layer-norm-like modulation. This design preserves the global consistency while allowing for flexible conditioning in dynamic scenarios. To ensure the quality of local dynamics essential in modulating BOV tokens, we further propose a refinement strategy for stream denoising. It decouples the sampling trajectory length from the attention window size constraint, leading to improved local guidance and overall imaging quality. We also propose a disturbance-augmented training noise schedule, which balances the convergence speed with model robustness for the stream denoising. Extensive experiments demonstrate that our method achieves compelling qualitative and quantitative results across multiple metrics.

视频生成扩散模型自回归动态一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。