利用视频编码中的运动矢量，无监督提升图像间单应性估计精度。
CodingHomo: Bootstrapping Deep Homography With Video Coding
- 从视频编码的运动矢量中提取特征，构建无监督单应性估计框架。
- 通过掩码引导融合模块提升特征有效性，使误差降低至2.7像素。
- 适合需要低资源、高鲁棒性的视觉定位与拼接场景使用。
单应性估计是计算机视觉的基础任务，广泛应用于多个领域。近年来深度学习的发展推动了无监督方法的进步，提升了模型的鲁棒性和泛化能力。然而，在复杂运动下准确预测单应性仍是挑战。本文提出 CodingHomo，一种基于视频编码的无监督单应性估计框架，充分利用视频中固有的运动矢量（Motion Vectors, MVs）。引入掩码引导融合（MGF）模块，筛选并强化有用的运动矢量特征；同时设计掩码引导单应性估计（MGHE）模块，在粗到精优化过程中去除冗余特征。在 Homography3D 与 Fly-Scenes 等数据集上，CodingHomo 在无监督条件下达到 2.7 像素平均误差，优于现有最优方法，展现出良好鲁棒性与泛化能力。代码与数据集已公开于 GitHub。
原文摘要 · Abstract (English)
Homography estimation is a fundamental task in computer vision with applications in diverse fields. Recent advances in deep learning have improved homography estimation, particularly with unsupervised learning approaches, offering increased robustness and generalizability. However, accurately predicting homography, especially in complex motions, remains a challenge. In response, this work introduces a novel method leveraging video coding, particularly by harnessing inherent motion vectors (MVs) present in videos. We present CodingHomo, an unsupervised framework for homography estimation. Our framework features a Mask-Guided Fusion (MGF) module that identifies and utilizes beneficial features among the MVs, thereby enhancing the accuracy of homography prediction. Additionally, the Mask-Guided Homography Estimation (MGHE) module is presented for eliminating undesired features in the coarse-to-fine homography refinement process. CodingHomo outperforms existing state-of-the-art unsupervised methods, delivering good robustness and generalizability. The code and dataset are available at: \href{github}{https://github.com/liuyike422/CodingHomo
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。