统一视频帧插值与预测，任意时间点都能精准生成中间帧和后续帧。
Unified Arbitrary-Time Video Frame Interpolation and Prediction
- 用同一网络架构+任务通道编码，实现插值与预测统一建模。
- 在多个数据集上插值性能媲美顶尖模型，预测性能超越现有最佳方法。
- 适合需要高效处理多任务视频生成的开发者与研究者。
视频帧插值与预测分别用于生成现有帧之间的中间帧和后续帧。尽管两者密切相关，传统方法常采用不同网络结构或独立训练权重。此外，任意时间插值已广泛研究，但任意时间预测的价值却长期被忽视。本文提出 uniVIP——统一的任意时间视频插值与预测框架。技术上，首先将仅支持插值的网络扩展为可处理插值与预测任务，并引入特殊输入通道编码任务类型；随后，展示如何在常见三帧序列上联合训练统一模型。实验表明，uniVIP 在视频插值任务上达到竞争性性能，在视频预测任务上超越现有最先进方法。代码将公开于：https://github.com/srcn-ivl/uniVIP。
原文摘要 · Abstract (English)
Video frame interpolation and prediction aim to synthesize frames in-between and subsequent to existing frames, respectively. Despite being closely-related, these two tasks are traditionally studied with different model architectures, or same architecture but individually trained weights. Furthermore, while arbitrary-time interpolation has been extensively studied, the value of arbitrary-time prediction has been largely overlooked. In this work, we present uniVIP - unified arbitrary-time Video Interpolation and Prediction. Technically, we firstly extend an interpolation-only network for arbitrary-time interpolation and prediction, with a special input channel for task (interpolation or prediction) encoding. Then, we show how to train a unified model on common triplet frames. Our uniVIP provides competitive results for video interpolation, and outperforms existing state-of-the-arts for video prediction. Codes will be available at: https://github.com/srcn-ivl/uniVIP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。