让视频生成支持任意时间顺序,可前后扩展或补全中间帧。
UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation

- 引入块级锚点潜变量,解决反向生成时的上下文断层问题。
- 单模型实现双向与中间帧生成,长视频质量接近仅正向模型。
- 适合需要灵活控制生成顺序的创意创作场景。
自回归视频扩散模型在长视频生成中表现出色,但现有方法仅支持正向生成。实际视频创作常需灵活顺序,如基于未来帧反向扩展,或结合前后帧补全中间内容。本文提出UniTemp,通过双向蒸馏框架训练单一自回归模型,支持任意时间方向生成。核心挑战在于3D VAE的因果结构导致反向生成时块间出现断层。为此,引入块级锚点潜变量,在反向生成时恢复块边界缺失的过去上下文。实验表明,UniTemp在短/长视频生成上性能与仅正向模型相当,同时支持双向扩展、中间帧生成、循环视频、场景过渡和视觉故事生成等多种工作流。
原文摘要 · Abstract (English)
Autoregressive video diffusion models have emerged as a promising approach for long video generation, achieving strong performance in streaming settings. However, existing methods are restricted to forward temporal generation, whereas practical video creation often requires flexible generation order, e.g., conditioning on future context to extend backward, or on both past and future context for inbetween generation. We bridge this gap by training an autoregressive model that supports generation in arbitrary temporal directions. A key technical challenge arises from the Causal 3D VAE widely used in video diffusion models, which encodes latents strictly conditioned on past context. While suited for forward generation, this causal structure causes inter-block discontinuities when generation proceeds backward. To address this, we introduce blockwise anchor latents, a set of auxiliary latents that restore the missing past context at block boundaries during backward generation. Built on this design, we propose UniTemp, a bidirectional distillation framework that trains a single autoregressive student model for any-direction video generation. At inference time, UniTemp conditions on arbitrary past and/or future frames, improving controllability for both bidirectional and inbetween generation. Experiments show that UniTemp maintains competitive performance on short and long video generation compared to forward-only methods, while enabling diverse workflows such as bidirectional video extension, inbetween generation, looping video generation, scene transition, and visual story generation. Project website: https://lzhangbj.github.io/projects/unitemp/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。