用视觉大模型语义信息增强局部特征匹配，提升定位精度
Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence
- 引入DINOv2语义提示改进特征描述子，实现无图像对推理
- 相机定位平均性能提升29%，媲美LightGlue和LoFTR
- 支持特征缓存与相似性搜索，适合高速匹配场景
视觉对应是相机定位、图像配准和运动结构等关键计算机视觉任务的核心步骤。当前最有效的关键点匹配方法依赖于学习的稀疏或稠密匹配器，但这些方法需成对图像输入，且在不同语义区域间匹配能力弱。本文提出一种新方法，利用基础视觉模型（如DINOv2）的语义线索，将语义推理融入现有特征描述子中，使学习到的描述子在推理时无需图像对，可实现特征缓存与快速相似性搜索。我们改进了六种现有描述子，在两个基准测试中相机定位平均性能提升29%，与LightGlue和LoFTR相当。代码与训练模型已公开。
原文摘要 · Abstract (English)
Visual correspondence is a crucial step in key computer vision tasks, including camera localization, image registration, and structure from motion. The most effective techniques for matching keypoints currently involve using learned sparse or dense matchers, which need pairs of images. These neural networks have a good general understanding of features from both images, but they often struggle to match points from different semantic areas. This paper presents a new method that uses semantic cues from foundation vision model features (like DINOv2) to enhance local feature matching by incorporating semantic reasoning into existing descriptors. Therefore, the learned descriptors do not require image pairs at inference time, allowing feature caching and fast matching using similarity search, unlike learned matchers. We present adapted versions of six existing descriptors, with an average increase in performance of 29% in camera localization, with comparable accuracy to existing matchers as LightGlue and LoFTR in two existing benchmarks. Both code and trained models are available at https://www.verlab.dcc.ufmg.br/descriptors/reasoning_accv24
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。