用视觉变压器联合训练位置描述符和图像对分类器，提升定位精度。
Pair-VPR: Place-Aware Pre-training and Contrastive Pair Classification for Visual Place Recognition with Vision Transformers
- 采用双分支视觉变压器结构，同时学习全局特征与图像对匹配判断。
- 在五个基准数据集上达到顶尖性能，大模型下定位召回率进一步提升。
- 基于场景感知采样预训练，让模型更适配视觉位置识别任务。
本文提出一种新的视觉位置识别（VPR）联合训练方法，同时学习全局描述符和用于重排序的图像对分类器。该分类器可判断一对图像是否来自同一位置。网络仅由视觉变压器组件构成，编码器与分类器均使用各自的任务令牌进行训练。不同于传统方法使用ImageNet等通用图像数据预训练，本文提出采用孪生掩码图像建模作为预训练任务，并设计场景感知的图像采样策略，从大规模VPR数据集中提取样本以优化模型。通过在第二阶段复用掩码建模的编码器和解码器权重，Pair-VPR在使用ViT-B编码器时，于五个基准数据集上实现最优表现，且在更大编码器下进一步提升定位召回率。
原文摘要 · Abstract (English)
In this work we propose a novel joint training method for Visual Place Recognition (VPR), which simultaneously learns a global descriptor and a pair classifier for re-ranking. The pair classifier can predict whether a given pair of images are from the same place or not. The network only comprises Vision Transformer components for both the encoder and the pair classifier, and both components are trained using their respective class tokens. In existing VPR methods, typically the network is initialized using pre-trained weights from a generic image dataset such as ImageNet. In this work we propose an alternative pre-training strategy, by using Siamese Masked Image Modelling as a pre-training task. We propose a Place-aware image sampling procedure from a collection of large VPR datasets for pre-training our model, to learn visual features tuned specifically for VPR. By re-using the Mask Image Modelling encoder and decoder weights in the second stage of training, Pair-VPR can achieve state-of-the-art VPR performance across five benchmark datasets with a ViT-B encoder, along with further improvements in localization recall with larger encoders. The Pair-VPR website is: https://csiro-robotics.github.io/Pair-VPR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。