不用递归生成，用扩散模型直接预测未来视频帧
Autoregression-free video prediction using diffusion model for mitigating error propagation
- 跳过逐帧预测，直接从上下文帧生成未来帧组合
- 在两个基准数据集上超越现有最优方法
- 适合需要减少误差累积的长期视频预测场景
现有的长期视频预测方法通常依赖自回归机制，但该方法在远期帧中容易产生误差传播。为此，本文首次提出基于扩散模型的无自回归（ARFree）视频预测框架。与传统自回归方式不同，ARFree可直接从上下文帧组预测任意未来帧组。该框架包含两个关键组件：1）运动预测模块，利用上下文帧组提取的运动特征预测未来运动；2）一种训练方法，提升相邻未来帧组之间的运动连续性与上下文一致性。在两个基准数据集上的实验表明，所提的ARFree框架优于多个当前最先进的视频预测方法。
原文摘要 · Abstract (English)
Existing long-term video prediction methods often rely on an autoregressive video prediction mechanism. However, this approach suffers from error propagation, particularly in distant future frames. To address this limitation, this paper proposes the first AutoRegression-Free (ARFree) video prediction framework using diffusion models. Different from an autoregressive video prediction mechanism, ARFree directly predicts any future frame tuples from the context frame tuple. The proposed ARFree consists of two key components: 1) a motion prediction module that predicts a future motion using motion feature extracted from the context frame tuple; 2) a training method that improves motion continuity and contextual consistency between adjacent future frame tuples. Our experiments with two benchmark datasets show that the proposed ARFree video prediction framework outperforms several state-of-the-art video prediction methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。