用分割模型建模多视角共同可见区域,提升特征匹配鲁棒性
SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching

- 基于SAM构建跨视图交互机制,预测共可见区域与边界框
- 在多个基准上显著优于现有方法,尤其在视角和尺度变化大时
- 适合需要高鲁棒性匹配的三维重建与视觉定位任务
可靠对应关系估计是图像处理中的基础问题,支撑着结构光恢复、视觉定位和图像配准等应用。现有基于学习的方法虽提升了局部特征表示,但多数仍局限于像素或小块级别,缺乏对多视角共同可见区域的显式建模。本文提出SAMatcher,一种通过共可见性建模进行对应关系估计的特征匹配框架。不直接匹配局部特征,而是先预测共可见区域掩码和边界框作为对应关系的结构先验。基于段落任意模型(SAM),引入对称跨视图交互机制,实现双向特征交换与跨视图语义对齐。进一步设计统一监督方案,通过掩码学习、边界框回归及掩码-边界框一致性约束联合优化掩码预测与边界框定位。大量实验证明,该方法在具有挑战性的基准上显著优于现有匹配流程,尤其在大视角和尺度变化下表现优异。结果表明,原本为单目分割设计的基础模型可通过显式共可见性建模有效拓展至多视图对应推理,为图像匹配的结构化表征学习提供了新视角。代码与项目页:https://xupan.top/Projects/samatcher
原文摘要 · Abstract (English)
Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。