用运动信息引导可变形卷积，提升视频帧预测精度。
FG-DFPN: Flow Guided Deformable Frame Prediction Network
- 用光流指导可变形采样，动态适应复杂运动。
- 在8个MPEG测试集上提升1dB PSNR，保持高速推理。
- 适合需要高保真时序预测的视频系统开发。
视频帧预测是计算机视觉中的基础挑战，对自动驾驶、视频压缩和媒体合成具有重要意义。本文提出FG-DFPN，一种融合光流估计与可变形卷积的新架构，通过运动线索引导可变形采样，解决固定卷积核在处理多样运动模式时的局限性。多尺度设计使模型能同时精确捕捉全局场景变换与局部物体运动。实验表明，FG-DFPN在8个多样化的MPEG测试序列上达到领先性能，相比现有方法提升1dB PSNR，且推理速度具有竞争力。将运动线索与自适应几何变换结合，使FG-DFPN成为下一代高保真视频处理系统的有力候选。代码与复现说明将发布于：https://github.com/KUIS-AI-Tekalp-Research Group/frame-prediction
原文摘要 · Abstract (English)
Video frame prediction remains a fundamental challenge in computer vision with direct implications for autonomous systems, video compression, and media synthesis. We present FG-DFPN, a novel architecture that harnesses the synergy between optical flow estimation and deformable convolutions to model complex spatio-temporal dynamics. By guiding deformable sampling with motion cues, our approach addresses the limitations of fixed-kernel networks when handling diverse motion patterns. The multi-scale design enables FG-DFPN to simultaneously capture global scene transformations and local object movements with remarkable precision. Our experiments demonstrate that FG-DFPN achieves state-of-the-art performance on eight diverse MPEG test sequences, outperforming existing methods by 1dB PSNR while maintaining competitive inference speeds. The integration of motion cues with adaptive geometric transformations makes FG-DFPN a promising solution for next-generation video processing systems that require high-fidelity temporal predictions. The model and instructions to reproduce our results will be released at: https://github.com/KUIS-AI-Tekalp-Research Group/frame-prediction
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。