arXiv:2604.13941cs.CV2026-04

让特征匹配学会看全局场景,无需标注就能更准地找对应点。

SceneGlue: Scene-Aware Transformer for Feature Matching without Scene-Level Annotation

论文配图:SceneGlue: Scene-Aware Transformer for Feature Matching without Scene-Level Annotation
图 1 · 摘自论文原文
  • 用注意力机制融合跨图像局部特征,建模全局场景上下文。
  • 在多个任务上显著提升匹配精度,如单应性估计误差降低17%。
  • 不依赖场景级标注,适合实际应用中缺乏标签的场景。

局部特征匹配在理解多视角图像间对应关系中起关键作用。传统方法受限于特征描述子的局部特性,难以捕捉对准确对应至关重要的非局部场景信息。本文提出SceneGlue,一种场景感知的特征匹配框架,通过可组合的匹配范式,融合隐式并行注意力与显式跨视角可见性估计。并行注意力机制在图像内和跨图像间同时交换局部描述子信息,增强场景全局上下文。为进一步提升场景感知能力,提出可见性Transformer,显式区分特征的可见与不可见区域,明确跨视角场景可见性。通过结合显式与隐式场景意识,SceneGlue有效弥补局部描述子的局限。值得注意的是,SceneGlue仅使用局部特征匹配进行训练,无需场景级真实标注。该场景感知方法不仅提升了准确性与鲁棒性,还增强了可解释性。在单应性估计、位姿估计、图像匹配与视觉定位等任务上的大量实验验证了其优越性能。源代码已公开于https://github.com/songlin-du/SceneGlue。

原文摘要 · Abstract (English)

Local feature matching plays a critical role in understanding the correspondence between cross-view images. However, traditional methods are constrained by the inherent local nature of feature descriptors, limiting their ability to capture non-local scene information that is essential for accurate cross-view correspondence. In this paper, we introduce SceneGlue, a scene-aware feature matching framework designed to overcome these limitations. SceneGlue leverages a hybridizable matching paradigm that integrates implicit parallel attention and explicit cross-view visibility estimation. The parallel attention mechanism simultaneously exchanges information among local descriptors within and across images, enhancing the scene's global context. To further enrich the scene awareness, we propose the Visibility Transformer, which explicitly categorizes features into visible and invisible regions, providing an understanding of cross-view scene visibility. By combining explicit and implicit scene-level awareness, SceneGlue effectively compensates for the local descriptor constraints. Notably, SceneGlue is trained using only local feature matches, without requiring scene-level groundtruth annotations. This scene-aware approach not only improves accuracy and robustness but also enhances interpretability compared to traditional methods. Extensive experiments on applications such as homography estimation, pose estimation, image matching, and visual localization validate SceneGlue's superior performance. The source code is available at https://github.com/songlin-du/SceneGlue.

特征匹配场景感知注意力机制无监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。