深度学习特征匹配显著优于传统方法，提升三维建筑模型定位精度。
To Glue or Not to Glue? Classical vs Learned Image Matching for Mobile Mapping Cameras to Textured Semantic 3D Building Models
- 用深度学习特征匹配替代传统SIFT+RANSAC方法
- 在无内点情况下仍保持高定位精度，误差低于10厘米
- 适合移动端建筑模型视觉定位场景
特征匹配是图像配准、运动结构重建和视觉定位等计算机视觉与摄影测量应用的关键步骤。传统手工设计方法如SIFT特征检测与描述结合最近邻匹配及RANSAC outlier去除曾为移动测绘相机的主流方案。随着深度学习发展，可学习特征匹配方法展现出更强鲁棒性与性能。然而，针对语义3D建筑模型（纹理化CityGML LoD2）中相机到模型匹配任务，经典与可学习方法的系统对比仍缺乏。本研究在标准数据集（HPatches、MegaDepth-1500）与自建数据集（立面纹理与对应地面/无人机图像）上评估多种特征匹配技术在视觉定位中的表现。使用基于透视n点（PnP）算法估计绝对姿态，并以地理参考轨迹数据作为几何真值。结果表明，在挑战性自建数据集上，可学习方法在零至12个RANSAC内点、零至0.16面积曲线下表现远超传统方法，定位精度显著提升。本工作将推动基于模型的视觉定位方法发展。
原文摘要 · Abstract (English)
Feature matching is a necessary step for many computer vision and photogrammetry applications such as image registration, structure-from-motion, and visual localization. Classical handcrafted methods such as SIFT feature detection and description combined with nearest neighbour matching and RANSAC outlier removal have been state-of-the-art for mobile mapping cameras. With recent advances in deep learning, learnable methods have been introduced and proven to have better robustness and performance under complex conditions. Despite their growing adoption, a comprehensive comparison between classical and learnable feature matching methods for the specific task of semantic 3D building camera-to-model matching is still missing. This submission systematically evaluates the effectiveness of different feature-matching techniques in visual localization using textured CityGML LoD2 models. We use standard benchmark datasets (HPatches, MegaDepth-1500) and custom datasets consisting of facade textures and corresponding camera images (terrestrial and drone). For the latter, we evaluate the achievable accuracy of the absolute pose estimated using a Perspective-n-Point (PnP) algorithm, with geometric ground truth derived from geo-referenced trajectory data. The results indicate that the learnable feature matching methods vastly outperform traditional approaches regarding accuracy and robustness on our challenging custom datasets with zero to 12 RANSAC-inliers and zero to 0.16 area under the curve. We believe that this work will foster the development of model-based visual localization methods. Link to the code: https://github.com/simBauer/To\_Glue\_or\_not\_to\_Glue
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。