arXiv:2606.03971cs.CV2026-06被引 1

让视频生成模型提前看未来,提升长序列一致性。

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

论文配图:Video-Mirai: Autoregressive Video Diffusion Models Need Foresight
图 1 · 摘自论文原文
  • 训练时用未来帧反向优化当前表示,不改变推理结构。
  • 5秒视频生成得分从83.8升至84.6,30秒一致性显著提升。
  • 适合追求长视频连贯性的扩散模型研究者使用。

因果视频生成器需从过去预测未来,但不应仅依赖过去信息。在自回归视频扩散模型中,每生成一段即形成对未来的承诺,而标准训练仅要求当前状态解释现时刻内容,导致表征层面的规划缺口:当前状态可能丢失身份、布局与运动等未来所需信息。本文提出Video-Mirai,一种仅用于训练的方法:模型按因果方式生成,冻结的前瞻编码器非因果读取完整生成序列,轻量级预测器将停止梯度的目标信息蒸馏回因果状态。未来帧监督表征,而非生成输入。推理时移除编码器与预测器,保持原架构、每步计算量及键值缓存行为不变。在5秒VBench测试中,总分从83.8提升至84.6;30秒超出训练时长的生成中,主体一致性由84.9升至88.5,背景一致性从90.2增至91.9。消融实验确认未来条件目标为关键因素,探针显示当前特征对未来的可解码性增强。结论:因果应约束推理,而非表征监督。视觉自回归模型需要前瞻性。项目页面:https://y0uroy.github.io/Video-Mirai。

原文摘要 · Abstract (English)

Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segment becomes a commitment that future segments must preserve. Standard training, however, only asks each causal state to explain the present. This creates what we call a representation-level planning gap: states that fit the current segment may discard identity, layout, and motion information needed for a consistent future. We introduce Video-Mirai, a training-only method that closes this gap without changing causal inference: the generator rolls out causally, a frozen foresight encoder reads the completed rollout non-causally, and a lightweight predictor distills the resulting stopped-gradient targets into causal states. Future frames supervise representations, never generator inputs. At inference, the encoder and predictor are discarded, leaving the original architecture, per-step FLOPs, and KV-cache behavior unchanged. Video-Mirai improves a strong Causal-Forcing baseline on 5-second VBench from 83.8 to 84.6 in terms of Total Score. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.9 to 88.5 and background consistency from 90.2 to 91.9. Ablations identify future-conditioned targets as the key ingredient, and probes show that future frames become more decodable from current features. Causality should constrain inference, not representation supervision. Our study highlights that visual autoregressive models need foresight. Project page: https://y0uroy.github.io/Video-Mirai.

视频生成扩散模型自回归前瞻性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。