用扩散模型提升真实图像退化下的光流估计精度
DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models
- 融合扩散模型中间特征与卷积特征，实现跨帧时序感知
- 在严重退化条件下，多个基准上显著优于现有方法
- 无需微调即可零样本适应真实世界噪声和模糊
基于高质量数据训练的光流模型在面对真实世界退化（如模糊、噪声、压缩伪影）时性能急剧下降。为此，我们提出「退化感知光流」新任务，旨在从真实退化视频中准确估计密集对应关系。核心洞察是：图像修复扩散模型的中间表示具备退化感知能力，但缺乏时序感知。为此，我们引入全时空注意力机制，使模型能够跨相邻帧建模，实证表明由此产生的特征具备零样本对应能力。基于此，我们构建了DA-Flow——一种将扩散特征与卷积特征融合于迭代优化框架中的混合架构。在多个基准上，该方法在严重退化条件下显著超越现有光流方法。
原文摘要 · Abstract (English)
Optical flow models trained on high-quality data often degrade severely when confronted with real-world corruptions such as blur, noise, and compression artifacts. To overcome this limitation, we formulate Degradation-Aware Optical Flow, a new task targeting accurate dense correspondence estimation from real-world corrupted videos. Our key insight is that the intermediate representations of image restoration diffusion models are inherently corruption-aware but lack temporal awareness. To address this limitation, we lift the model to attend across adjacent frames via full spatio-temporal attention, and empirically demonstrate that the resulting features exhibit zero-shot correspondence capabilities. Based on this finding, we present DA-Flow, a hybrid architecture that fuses these diffusion features with convolutional features within an iterative refinement framework. DA-Flow substantially outperforms existing optical flow methods under severe degradation across multiple benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。