arXiv:2502.16601cs.CV2025-02TPAMI被引 16

SelaVPR++用轻量适配器和二值特征,让视觉定位更高效准确

SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efficient Place Recognition

  • 用轻量多尺度卷积适配器,在不反向传播的情况下优化特征
  • 采用二值特征初检+浮点特征重排,降低延迟和存储开销
  • 适合追求高效率的实时定位场景,尤其在资源受限设备上

近期研究显示,使用预训练视觉基础模型的视觉位置识别(VPR)方法可取得优异性能。在之前的工作中,我们提出SelaVPR,通过参数高效适配实现基础模型到VPR的无缝适配,生成聚焦显著地标的同时包含全局与局部特征,用于两阶段VPR。尽管如此,我们仍认为先前适配在训练时间和GPU内存使用上效率不足,且重排范式在检索延迟与存储方面成本较高。为此,我们提出SelaVPR++的改进版本。具体而言,设计了一种参数、时间与内存高效的适配方法,采用轻量级多尺度卷积(MultiConv)适配器对冻结的基础骨干网络中间特征进行精细化调整。该方法在训练时不向骨干网络反向传播梯度,且多尺度卷积促进空间轴上的特征交互并引入合适局部先验,从而实现更高效率与更好性能。此外,提出一种创新的重排范式以提升效率:不再依赖局部特征进行重排,而是使用紧凑的二值特征进行初始检索,再以鲁棒的浮点(全局)特征进行重排。为获得此类二值特征,提出一种相似性约束深度哈希方法,可轻松集成至VPR流程。最后,改进训练策略,并统一多个常用训练数据集的训练协议,合并训练以增强模型泛化能力。大量实验表明……

原文摘要 · Abstract (English)

Recent studies show that the visual place recognition (VPR) method using pre-trained visual foundation models can achieve promising performance. In our previous work, we propose a novel method to realize seamless adaptation of foundation models to VPR (SelaVPR). This method can produce both global and local features that focus on discriminative landmarks to recognize places for two-stage VPR by a parameter-efficient adaptation approach. Although SelaVPR has achieved competitive results, we argue that the previous adaptation is inefficient in training time and GPU memory usage, and the re-ranking paradigm is also costly in retrieval latency and storage usage. In pursuit of higher efficiency and better performance, we propose an extension of the SelaVPR, called SelaVPR++. Concretely, we first design a parameter-, time-, and memory-efficient adaptation method that uses lightweight multi-scale convolution (MultiConv) adapters to refine intermediate features from the frozen foundation backbone. This adaptation method does not back-propagate gradients through the backbone during training, and the MultiConv adapter facilitates feature interactions along the spatial axes and introduces proper local priors, thus achieving higher efficiency and better performance. Moreover, we propose an innovative re-ranking paradigm for more efficient VPR. Instead of relying on local features for re-ranking, which incurs huge overhead in latency and storage, we employ compact binary features for initial retrieval and robust floating-point (global) features for re-ranking. To obtain such binary features, we propose a similarity-constrained deep hashing method, which can be easily integrated into the VPR pipeline. Finally, we improve our training strategy and unify the training protocol of several common training datasets to merge them for better training of VPR models. Extensive experiments show that ......

视觉定位基础模型高效推理特征哈希

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。