首个融合多视角3D信息的通用视觉定位模型，提升跨环境识别能力。
UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformer
- 基于多视角3D特征编码器，联合利用2D与3D token构建描述符
- 在10个数据集上达到新最优，多视角设置下平均mAP提升5.2%
- 适合需要高鲁棒性的自动驾驶与机器人定位场景
传统视觉定位（VPR）通常作为单图像检索任务处理。多视角输入虽具优势，但该设定仍研究不足，现有方法在多样环境中泛化能力有限。本文提出UniPR-3D，首个有效整合多视角信息的VPR架构。其基于可编码多视图3D表示的VGGT主干网络，通过设计特征聚合模块并微调以适配定位任务。通过联合利用VGGT产生的3D token与中间2D token，分别设计专用聚合模块，使描述符既能捕捉细微纹理线索，又能跨视角推理。为进一步提升泛化性，引入单帧与多帧聚合策略，以及可变长度序列检索机制。实验表明，UniPR-3D在10个数据集上均优于单/多视角基线，刷新当前最优性能，验证了几何感知token在VPR中的有效性。代码与模型将开源至GitHub。
原文摘要 · Abstract (English)
Visual Place Recognition (VPR) has been traditionally formulated as a single-image retrieval task. Using multiple views offers clear advantages, yet this setting remains relatively underexplored and existing methods often struggle to generalize across diverse environments. In this work we introduce UniPR-3D, the first VPR architecture that effectively integrates information from multiple views. UniPR-3D builds on a VGGT backbone capable of encoding multi-view 3D representations, which we adapt by designing feature aggregators and fine-tune for the place recognition task. To construct our descriptor, we jointly leverage the 3D tokens and intermediate 2D tokens produced by VGGT. Based on their distinct characteristics, we design dedicated aggregation modules for 2D and 3D features, allowing our descriptor to capture fine-grained texture cues while also reasoning across viewpoints. To further enhance generalization, we incorporate both single- and multi-frame aggregation schemes, along with a variable-length sequence retrieval strategy. Our experiments show that UniPR-3D sets a new state of the art, outperforming both single- and multi-view baselines and highlighting the effectiveness of geometry-grounded tokens for VPR. Our code and models will be made publicly available on Github https://github.com/dtc111111/UniPR-3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。