提出结构引导的Transformer,提升图像匹配中关键结构的关注度。
SGFormer: Structure-Guided Transformer for Robust Local Feature Matching

- 用结构引导注意力机制,聚焦重叠区域的关键特征
- 在大视角变化下显著减少错误匹配,准确率提升明显
- 适合3D重建、视觉定位等对匹配精度要求高的场景
局部特征匹配是摄影测量中的基础环节,对三维重建、立体映射和视觉定位等任务至关重要。尽管近期无检测器的匹配方法(如LoFTR)已取得进展,但其依赖全局注意力机制所获取的全局特征,在某些情况下会弱化对显著结构的关注,导致注意力发散现象:部分高置信度匹配分布在有效匹配区域之外,尤其在视角变化大的场景中更为明显。这是由于标准Transformer对无关区域中的相似特征给予同等权重,限制了在复杂摄影测量环境下的匹配可靠性。为此,本文提出SGFormer(Structure-Guided Transformer),一种结构感知的匹配网络,可自适应地增强重叠区域内显著结构附近的特征注意力。SGFormer采用半密集的粗到精流程,并在主干网络中引入提出的三重结构注意力(TSA)模块,利用浅层网络中的局部特征增强显著结构周围的表征,引导后续Transformer阶段在整个全局范围内加强对显著结构区域的关注,从而强化对视觉一致区域的注意力并抑制非重叠区域的影响。大量实验表明,SGFormer能显著缓解注意力发散问题,提升匹配精度。
原文摘要 · Abstract (English)
Local feature matching is a fundamental component of photogrammetry, enabling accurate image correspondence critical for tasks such as 3D reconstruction, stereo mapping, and visual localization. While recent detector-free matching methods, like LoFTR, have advanced the field, the global features obtained by leveraging the global-range modeling capacity of the unconstrained attention mechanism compromise the model's attention to the salient structures in certain scenarios. This limitation leads to a phenomenon we define as attention divergence, wherein a portion of high-confidence matches are distributed outside the valid matching region (overlapping region), especially in scenes with large viewpoint variations. This occurs because similar features in irrelevant regions may receive equal weighting and consideration within the standard Transformer, limiting matching reliability in challenging photogrammetric environments. To address this issue in feature matching, we propose SGFormer (Structure-Guided Transformer), a novel structure-aware matching network that adaptively updates attention on features near salient structure in overlapping regions. SGFormer employs a semi-dense coarse-to-fine pipeline and incorporates the proposed Triple-Structure-Attention (TSA) module into the backbone net for extracting distinctive features. The TSA module utilizes shallow local features from early network layers to enhance the representation around salient structure, guiding subsequent transformer stages to intensify the model's focus on regions with salient structure across the global scope. SGFormer, thereby reinforcing attention to visually consistent areas while mitigating the influence of non-overlapping regions. Extensive experiments show that SGFormer significantly mitigates attention divergence and improves matching accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。