首个通过早期融合实现多视角空间整合的视觉定位框架,提升复杂环境鲁棒性。
Reloc-VGGT: Visual Re-localization with Geometry Grounded Transformer
- 采用早期融合机制整合多视角3D几何信息,替代传统后期平均
- 在八百万图像对上训练,实现在多样数据集上的实时高精度定位
- 引入稀疏掩码注意力,降低计算开销,适合大规模部署
视觉定位传统上被建模为成对姿态回归问题。现有方法主要估计两图之间的相对姿态,并采用后期融合策略获得绝对姿态估计。然而,后期运动平均往往无法有效整合空间信息,且在复杂环境中精度下降。本文提出首个通过早期融合机制实现多视角空间整合的视觉定位框架,可在结构化与非结构化环境中稳定运行。框架基于VGGT主干网络,编码多视角3D几何信息,并引入姿态分词器与投影模块,更有效地利用多数据库视角间的空间关系。此外,提出一种新型稀疏掩码注意力策略,避免全局注意力的二次复杂度,实现大规模下的实时性能。在约八百万带姿态图像对上训练,Reloc-VGGT展现出强准确性与卓越泛化能力。跨多个公开数据集的大量实验一致验证了方法的有效性与效率,在实时生成高质量相机姿态的同时,对未见环境保持鲁棒性。代码与模型将在论文接受后公开。
原文摘要 · Abstract (English)
Visual localization has traditionally been formulated as a pair-wise pose regression problem. Existing approaches mainly estimate relative poses between two images and employ a late-fusion strategy to obtain absolute pose estimates. However, the late motion average is often insufficient for effectively integrating spatial information, and its accuracy degrades in complex environments. In this paper, we present the first visual localization framework that performs multi-view spatial integration through an early-fusion mechanism, enabling robust operation in both structured and unstructured environments. Our framework is built upon the VGGT backbone, which encodes multi-view 3D geometry, and we introduce a pose tokenizer and projection module to more effectively exploit spatial relationships from multiple database views. Furthermore, we propose a novel sparse mask attention strategy that reduces computational cost by avoiding the quadratic complexity of global attention, thereby enabling real-time performance at scale. Trained on approximately eight million posed image pairs, Reloc-VGGT demonstrates strong accuracy and remarkable generalization ability. Extensive experiments across diverse public datasets consistently validate the effectiveness and efficiency of our approach, delivering high-quality camera pose estimates in real time while maintaining robustness to unseen environments. Our code and models will be publicly released upon acceptance.https://github.com/dtc111111/Reloc-VGGT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。