通过3D几何信息提升2D特征描述子的鲁棒性，应对光照变化和低纹理场景。
LiftFeat: 3D Geometry-Aware Local Feature Matching
- 利用预训练单目深度模型生成法向量标签，指导3D几何特征提取。
- 设计几何感知融合模块，将法向量与2D描述子结合，增强区分能力。
- 在极端条件下优于主流轻量级方法，适合机器人视觉定位应用。
鲁棒高效的局部特征匹配在机器人领域的SLAM和视觉定位中至关重要。尽管进展显著，但在光照剧烈变化、低纹理区域或重复图案场景下，仍难以提取鲁棒且具有区分性的视觉特征。本文提出一种轻量级网络LiftFeat，通过聚合3D几何特征来提升原始描述子的鲁棒性。首先，采用预训练的单目深度估计模型生成伪表面法向量标签，以监督预测表面法向量的3D几何特征提取。随后设计3D几何感知特征提升模块，将表面法向量特征与原始2D描述子特征融合。引入3D几何信息显著增强了2D特征描述在极端条件下的区分能力。在相对位姿估计、单应性估计和视觉定位任务上的大量实验表明，LiftFeat优于部分轻量级先进方法。代码将发布于：https://github.com/lyp-deeplearning/LiftFeat。
原文摘要 · Abstract (English)
Robust and efficient local feature matching plays a crucial role in applications such as SLAM and visual localization for robotics. Despite great progress, it is still very challenging to extract robust and discriminative visual features in scenarios with drastic lighting changes, low texture areas, or repetitive patterns. In this paper, we propose a new lightweight network called \textit{LiftFeat}, which lifts the robustness of raw descriptor by aggregating 3D geometric feature. Specifically, we first adopt a pre-trained monocular depth estimation model to generate pseudo surface normal label, supervising the extraction of 3D geometric feature in terms of predicted surface normal. We then design a 3D geometry-aware feature lifting module to fuse surface normal feature with raw 2D descriptor feature. Integrating such 3D geometric feature enhances the discriminative ability of 2D feature description in extreme conditions. Extensive experimental results on relative pose estimation, homography estimation, and visual localization tasks, demonstrate that our LiftFeat outperforms some lightweight state-of-the-art methods. Code will be released at : https://github.com/lyp-deeplearning/LiftFeat.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。