提出加权聚合与动态剪枝,提升视觉定位精度与效率。
Faster or Stronger: Towards Flexible Visual Place Recognition via Weighted Aggregation and Token Pruning

- 加权聚合各特征块,提升全局描述符区分度
- 剪枝后推理速度提升3.2倍,精度仅降1.5%
- 支持推理时灵活调整精度与效率,适合边缘设备
视觉定位(VPR)旨在从大规模数据库中匹配查询图像与同一地点的参考图像。当前先进方法采用视觉变压器(ViT)提取对视角、光照和季节变化鲁棒的局部特征,并聚合为紧凑的全局描述符进行检索。现有聚合方法将特征块均匀聚类,但不同聚类编码的空间或语义模式差异大,贡献不均。为此,我们提出加权聚合描述符(WeiAD),在聚合时为聚类分配权重,生成更具区分性的全局表示。除精度外,检索延迟是大规模部署和资源受限边缘设备的关键挑战。以往工作主要通过压缩全局描述符降低延迟,却忽视了基于ViT骨干网络的特征提取开销。因此,我们提出面向VPR的令牌剪枝框架WeiToP,通过自蒸馏减少特征提取成本:由聚合产生的令牌重要性监督一个轻量级剪枝模块,该模块连接至早期变压器层,实现推理时的动态剪枝。经一次联合训练后,WeiToP可在推理时即插即用地进行剪枝,灵活控制精度-效率权衡,无需额外训练。此外,WeiToP优于适配通用视觉任务的现有剪枝方法。
原文摘要 · Abstract (English)
Visual Place Recognition (VPR) aims to match a query image to reference images of the same place in a large-scale database. Recent state-of-the-art methods employ Vision Transformers (ViTs) as backbone foundation models to extract patch-level features that are robust to viewpoint, illumination, and seasonal variations, which are then aggregated into a compact global descriptor for retrieval. Most existing aggregation methods uniformly pool patch tokens into learned clusters, despite the fact that different clusters often encode distinct spatial or semantic patterns and contribute unequally to VPR performance. To address this limitation, we propose Weighted Aggregated Descriptor (WeiAD), which assigns weights to clusters during aggregation, producing more discriminative global representations. Beyond accuracy, retrieval latency is a critical concern for large-scale deployments and resource-constrained edge devices. Prior work mainly reduces latency by compressing global descriptors, while overlooking the cost of feature extraction, an issue exacerbated by ViT-based backbones. We therefore introduce WeiToP, a VPR-oriented token pruning framework that reduces feature extraction cost via self-distillation, where aggregation-induced token importance supervises a lightweight pruning module attached to an early transformer layer, enabling inference-time token pruning. After a single joint training phase, WeiToP enables plug-and-play token pruning at inference time, allowing flexible and on-demand control over the accuracy-efficiency trade-off without additional training. Moreover, WeiToP outperforms existing token pruning methods adapted from general vision tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。