arXiv:2608.22861cs.CV2026-08中稿 · ECCV

用运动引导序列重排,让模型更好处理视频插帧中的动态运动。

Following Motion for Sequential Modeling in Video Frame Interpolation

论文配图:Following Motion for Sequential Modeling in Video Frame Interpolation
图 1 · 摘自论文原文
  • 基于光流动态排序输入,使状态更新匹配运动轨迹
  • 在复杂运动上达到当前最优,尤其提升大位移场景表现
  • 适合需要高精度运动建模的视频生成与修复任务

状态空间模型(SSMs)因其线性计算复杂度下捕捉长程依赖的能力,成为视频帧插值(VFI)的有前景架构。然而,其预定义扫描顺序限制了对VFI中固有动态运动轨迹的建模效果。为此,我们提出面向视频帧插值的运动引导Mamba(MGMVFI),该模型针对VFI特性改造了选择性状态空间模型。MGMVFI引入运动引导序列化(MGS),利用光流定义自适应的一维输入顺序,使因果状态更新与语义相关令牌对齐,实现运动一致的特征传播,尤其适用于大范围动态运动。此外,为缓解不准确光流带来的特征表示不可靠问题,我们设计上下文合成模块,利用周围空间上下文进行鲁棒的跨帧特征融合。这些组件无缝集成于定制的Mamba架构中,并采用轻量级精修模块,在降低计算成本的同时增强局部细节重建。在标准VFI基准上的大量实验表明,MGMVFI在复杂动态运动场景下均达到当前最优性能,为视频插值中的序列建模开辟新方向。

原文摘要 · Abstract (English)

State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI), as they can capture long-range dependencies with linear computational complexity. However, their predefined scanning order limits their effectiveness in modeling the dynamic motion trajectories inherent in VFI problems. To tackle this challenge, we propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI. MGMVFI introduces Motion-Guided Serialization (MGS), which leverages optical flow to define a motion-adaptive 1D input order for the SSM. This aligns the causal state updates with semantically related tokens, enabling motion-consistent feature propagation, particularly for large and dynamic motions. Additionally, to mitigate the unreliable feature representations caused by inaccurate optical flow estimates, we introduce contextual synthesis that utilizes the surrounding spatial context for robust inter-frame feature synthesis. These components are seamlessly integrated within our tailored Mamba architecture, which also employs a lightweight refinement block to enhance local detail reconstruction at a reduced computational cost. Extensive experiments on standard VFI benchmarks demonstrate that MGMVFI achievesstate-of-the-artperformance,particularly on complex and dynamic motions, thereby establishing a new direction for sequence modeling in video interpolation.

视频插帧状态空间模型运动建模Mamba

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。