用可学习的令牌实现隐式聚合,无需专门聚合器即可获得鲁棒图像全局表示。
Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era
- 在Transformer块前添加可学习的聚合令牌,通过自注意力机制隐式融合特征
- 在多个VPR数据集上优于现有方法,效率更高,MSLS排行榜排名第一
- 适合关注高效视觉定位与模型简化的研究者
视觉场景识别(VPR)通常被视为一种特定的图像检索任务,核心在于将图像表示为全局描述符。过去十年中,主流方法(如NetVLAD)采用骨干网络加聚合器的范式:先提取图像块特征,再通过聚合器生成全局描述符。这一范式在CNN时代占据主导地位,并被广泛沿用于基于Transformer的模型。本文提出,在Transformer时代,专用聚合器并非必需,仅靠骨干网络即可获得鲁棒的全局描述符。具体而言,我们在特定Transformer块前插入可学习的聚合令牌,这些令牌与图像块令牌共同参与自注意力计算,通过内在的全局交互机制,隐式聚合块内有用信息至聚合令牌。最终仅取最后一层输出中的聚合令牌拼接作为全局表示。尽管该隐式聚合方法以极简方式实现鲁棒描述符,但令牌插入位置、数量及初始化策略仍需探索。为此,我们基于实验提出了最优插入策略和初始化方法。实验表明,该方法在多个VPR数据集上超越当前最佳方法,效率更高,且在MSLS挑战赛榜单排名第一。代码已开源。
原文摘要 · Abstract (English)
Visual place recognition (VPR) is typically regarded as a specific image retrieval task, whose core lies in representing images as global descriptors. Over the past decade, dominant VPR methods (e.g., NetVLAD) have followed a paradigm that first extracts the patch features/tokens of the input image using a backbone, and then aggregates these patch features into a global descriptor via an aggregator. This backbone-plus-aggregator paradigm has achieved overwhelming dominance in the CNN era and remains widely used in transformer-based models. In this paper, however, we argue that a dedicated aggregator is not necessary in the transformer era, that is, we can obtain robust global descriptors only with the backbone. Specifically, we introduce some learnable aggregation tokens, which are prepended to the patch tokens before a particular transformer block. All these tokens will be jointly processed and interact globally via the intrinsic self-attention mechanism, implicitly aggregating useful information within the patch tokens to the aggregation tokens. Finally, we only take these aggregation tokens from the last output tokens and concatenate them as the global representation. Although implicit aggregation can provide robust global descriptors in an extremely simple manner, where and how to insert additional tokens, as well as the initialization of tokens, remains an open issue worthy of further exploration. To this end, we also propose the optimal token insertion strategy and token initialization method derived from empirical studies. Experimental results show that our method outperforms state-of-the-art methods on several VPR datasets with higher efficiency and ranks 1st on the MSLS challenge leaderboard. The code is available at https://github.com/lu-feng/image.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。