用轻量解码器提升视觉定位准确率,不依赖复杂重排序。
EDTformer: An Efficient Decoder Transformer for Visual Place Recognition
- 用可学习查询解码深层特征,逐层聚合上下文信息
- 在RVLCDIP、CVUSA等数据集上超越单阶段方法,接近双阶段性能
- 结合DINOv2与低秩适配,高效增强特征鲁棒性,适合嵌入式部署
视觉位置识别(VPR)旨在通过检索大型地理标记数据库中视觉相似图像,确定查询图像的地理位置。现有方法多聚焦于主干网络提取的深层特征聚合,常采用CNN、MLP或池化层,对变换器解码器关注不足。本文提出高效解码器变换器(EDTformer),由多个简化解码块堆叠而成,配合两个线性层直接生成强健且具有区分性的全局表征。具体地,将深层特征作为键值,可学习参数作为查询,充分捕捉特征内部上下文关系,逐步解码并聚合有效特征输出全局表示。此外,为提升特征质量,采用基础模型DINOv2作为主干,并提出低秩并行适配(LoPA)方法,以低内存和参数开销渐进优化中间特征。实验表明,该方法不仅优于多个基准数据集上的单阶段VPR方法,还超越需额外重排序的双阶段方法,且计算成本更低。代码将开源。
原文摘要 · Abstract (English)
Visual place recognition (VPR) aims to determine the general geographical location of a query image by retrieving visually similar images from a large geo-tagged database. To obtain a global representation for each place image, most approaches typically focus on the aggregation of deep features extracted from a backbone through using current prominent architectures (e.g., CNNs, MLPs, pooling layer, and transformer encoder), giving little attention to the transformer decoder. However, we argue that its strong capability to capture contextual dependencies and generate accurate features holds considerable potential for the VPR task. To this end, we propose an Efficient Decoder Transformer (EDTformer) for feature aggregation, which consists of several stacked simplified decoder blocks followed by two linear layers to directly produce robust and discriminative global representations. Specifically, we do this by formulating deep features as the keys and values, as well as a set of learnable parameters as the queries. Our EDTformer can fully utilize the contextual information within deep features, then gradually decode and aggregate the effective features into the learnable queries to output the global representations. Moreover, to provide more powerful deep features for EDTformer and further facilitate the robustness, we use the foundation model DINOv2 as the backbone and propose a Low-rank Parallel Adaptation (LoPA) method to enhance its performance in VPR, which can refine the intermediate features of the backbone progressively in a memory- and parameter-efficient way. As a result, our method not only outperforms single-stage VPR methods on multiple benchmark datasets, but also outperforms two-stage VPR methods which add a re-ranking with considerable cost. Code will be available at https://github.com/Tong-Jin01/EDTformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。