arXiv:2608.07120cs.CV2026-08

提出多假设光流估计,解决模糊区域插帧时的伪影问题。

Multiple Hypothesis Flow Estimation for Video Frame Interpolation under Matching Ambiguity

论文配图:Multiple Hypothesis Flow Estimation for Video Frame Interpolation under Matching Ambiguity
图 1 · 摘自论文原文
  • 保留多个候选匹配,通过可靠性路由选择最优流
  • 在MA-HD和公开数据集上LPIPS与DISTS指标最优
  • 适合处理重复纹理、对称结构等模糊场景

基于光流的视频帧插值方法通常通过估计光流场,将两帧输入图像进行形变并融合生成中间帧。这些隐式光流场通常在无直接光流标注的情况下,通过图像级重建监督进行学习。但在存在重复或随机纹理、旋转对称结构,或快速运动导致模糊的区域中,单个查询点的匹配证据可能包含多个空间分离且相近的峰值。尽管真实中间帧提供间接监督,却无法唯一确定模糊区域中的潜在对应关系。当多个位置提供多个合理匹配时,单一光流估计器只能保留一个位移并舍弃其余候选,若所选匹配错误或与邻近像素不一致,则会从错误位置采样内容,导致鬼影、结构失真或模糊。为解决此问题,本文提出一种多假设光流估计框架,保留前K个候选对应关系,并通过可靠性引导的路由器每位置选择一个。每个假设从粗粒度匹配锚点出发,经锚点中心局部注意力独立精炼。帧合成基于选定的单一流-外观假设,而非候选运动的软加权组合。在自建的MA-HD基准及公开的VFI基准上实验表明,本方法在所有对比方法中取得最优的LPIPS与DISTS性能。

原文摘要 · Abstract (English)

Many flow-based video frame interpolation (VFI) methods synthesize an intermediate frame by estimating optical flow fields, warping the two input frames, and blending the warped observations. These latent flow fields are typically learned through image-level reconstruction supervision without direct flow annotations. In ambiguous regions containing repetitive or stochastic textures, rotating symmetric structures, or fast motion with blur, the matching evidence for a single query may contain multiple comparable and spatially separated peaks. Although the ground-truth intermediate frame provides indirect supervision, it may not uniquely identify the latent correspondence in ambiguous regions.When several locations provide multiple plausible matches, a single-flow estimator can retain only one displacement and discard the remaining candidates. If the selected match is incorrect or inconsistent with those of neighboring pixels, warping samples content from mismatched locations, producing ghosting, structural distortion, or blur.To address this limitation, we propose a multiple hypothesis flow estimation framework that preserves top-K candidate correspondences and selects one per location through a reliability-guided router. Each hypothesis is initialized from a coarse matching anchor and refined separately through anchor-centered local attention. Frame synthesis is thus conditioned on one selected flow-appearance hypothesis rather than a soft combination of candidate motions.Experiments on the proposed MA-HD benchmark and public VFI benchmarks show that our method achieves the best LPIPS and DISTS among the compared methods.

视频插帧光流估计多假设去伪影

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。