arXiv:2607.09503cs.CVcs.AI2026-07

VGGT模型能无监督识别图像是否重叠,效果超人类标注。

What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility

论文配图:What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility
图 1 · 摘自论文原文
  • 用VGGT提取图像特征,通过层间结构判断重叠性。
  • 在Co-VisiON上比前人提升25%以上,且预测概率准确。
  • 适合做3D重建与机器人定位中的可见性判断。

3D重建和机器人定位的核心挑战之一是共视性:判断图像对是否共享可见表面,尤其在重叠极少时。我们发现,尽管未针对该任务进行训练,VGGT的内部表征却自然涌现出共视性判断能力:早期层构建3D场景感知表示,晚期层则充当专门的共视性推理器。特别地,我们识别出第L17层作为负样本锚点,始终将非共视图像对导向该层,为几何基础模型中层特化提供了任务导向证据。基于此,我们提出Co-VGGT,冻结VGGT主干,仅训练一个轻量级分层专家混合头(参数少于7.5M),从单张RGB图像中分类共视性,将每层视为特定专家,按输入对自适应加权其几何抽象。在Co-VisiON基准上,Co-VGGT超越人工标注基线,相比先前方法实现超过25%的成对性能提升和10%的多视角提升。成对预测校准良好(ECE=0.030),可直接作为可见性图边权重用于后续SfM和SLAM流程,无需后处理修正。代码与数据已公开。

原文摘要 · Abstract (English)

A fundamental challenge in 3D reconstruction and robotic localization is co-visibility: determining which image pairs share overlapping visible surfaces, particularly in scenarios with minimal overlap. We demonstrate that VGGT implicitly encodes co-visibility as an emergent behavior: without any supervision for this task, its internal representations exhibit a clear hierarchical structure mirroring that of large language models, i.e. early layers build a 3D-aware scene representation, while late layers act as dedicated co-visibility reasoners. In particular, we identify layer L17 as a negative anchor that consistently routes non-co-visible pairs for this backbone, regardless of the evaluation setting, providing task-grounded evidence of layer specialization in a geometry-grounded foundation model. Building on this, we introduce Co-VGGT, which freezes VGGT and trains only a lightweight layer-wise mixture-of-experts head (less than 7.5M parameters) to classify co-visibility from RGB alone, treating each layer as a specialized expert whose geometric abstraction is adaptively weighted per input pair. On the Co-VisiON benchmark, Co-VGGT surpasses the human annotation baseline and improves over prior work by more than 25% pairwise and 10% multiview. Pairwise predictions are well-calibrated (ECE=0.030), enabling direct use as edge weights in visibility graphs for downstream SfM and SLAM pipelines without post-hoc correction. Code and data are available.

3D重建共视性视觉基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。