研究视觉定位中哪些图像片段可删减，提升效率又不损精度。
Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference
- 系统测试剪枝、合并等令牌压缩方法在视觉定位中的效果。
- 压缩后计算量降29%，吞吐量提44%，准确率损失不足1%。
- 适合想部署轻量级视觉定位系统的开发者参考。
基于视觉变压器的视觉定位（VPR）方法，特别是基础模型，已取得卓越识别性能。然而，这些模型需处理全部视觉令牌，导致显著计算开销，阻碍其在实时与资源受限场景中的应用。一个自然的问题是：所有视觉令牌对VPR是否必要？为此，我们首次系统性地构建了面向高效VPR的令牌压缩基准。该基准全面评估了代表性令牌剪枝、合并及混合方法，在多个先进的VPR模型和涵盖城市、郊区与自然环境的多样化数据集上进行测试。我们从不同角度分析令牌压缩，包括不同压缩配置下的识别性能、计算复杂度、推理速度、定性可视化以及边缘设备上的部署效率。通过大量实验与深入分析，我们的基准揭示了令牌压缩在VPR中的多个重要特性，并提供了关于精度与推理效率权衡的实用见解。例如，令牌压缩可使计算成本降低最多29%，吞吐量提升最高44%，同时识别准确率下降不足1%。总体而言，本工作为未来令牌高效的VPR及高效视觉检索系统研究奠定了综合基础。代码与模型将公开于https://github.com/Tong-Jin01/TokenReduction4VPR。
原文摘要 · Abstract (English)
Recent visual place recognition (VPR) methods based on vision transformers, particularly foundation models, have achieved remarkable recognition performance. However, these models process all visual tokens throughout the entire network, resulting in substantial computational overhead, which hinders their deployment in real-time and resource-constrained scenarios. A natural question thus arises: are all visual tokens necessary for VPR? To answer this question, we present the first systematic benchmark of token reduction for efficient visual place recognition. Our benchmark comprehensively evaluates representative token pruning, token merging, and hybrid pruning-merging methods across multiple state-of-the-art VPR models and diverse benchmark datasets covering urban, suburban, and natural environments. We further investigate token reduction from multiple perspectives, including recognition performance under different reduction configurations, computational complexity, inference speed, qualitative visualization, and deployment efficiency on edge devices. Through extensive experiments and in-depth analysis, our benchmark reveals multiple important characteristics of token reduction in VPR and provides several practical insights into the trade-offs between accuracy and inference efficiency. For example, token reduction can reduce computational cost by up to 29\% and improve throughput by up to 44\%, while incurring less than 1\% degradation in recognition accuracy. Overall, this work establishes a comprehensive foundation for future research on token-efficient VPR and efficient visual retrieval systems. Our codes and models will be available at https://github.com/Tong-Jin01/TokenReduction4VPR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。