只提取关键帧块，让视频行人重识别更高效
Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification
- 只选重要且不重复的图像块进行特征提取
- 计算量比ViT-B降低74%，准确率接近ViT-B
- 适合追求效率与精度平衡的视频ReID应用
本文提出一种新型高效可插拔的视频行人重识别骨干网络。传统方法对每帧中每个位置都进行深度特征提取，但研究发现，由于人体动作细微，视频中不同帧间差异小，存在大量重复区域。为此，本文设计了一种选择性特征提取机制，仅对关键非重复块进行处理，显著降低计算开销。同时引入伪帧全局上下文生成与利用模块，缓解稀疏输入导致的视图不完整问题。在多个数据集上的实验表明，该方法相比ViT-B减少74%计算量，相比ResNet50减少28%计算量，准确率与ViT-B相当，显著优于ResNet50。
原文摘要 · Abstract (English)
This paper proposes a new effective and efficient plug-and-play backbone for video-based person re-identification (ReID). Conventional video-based ReID methods typically use CNN or transformer backbones to extract deep features for every position in every sampled video frame. Here, we argue that this exhaustive feature extraction could be unnecessary, since we find that different frames in a ReID video often exhibit small differences and contain many similar regions due to the relatively slight movements of human beings. Inspired by this, a more selective, efficient paradigm is explored in this paper. Specifically, we introduce a patch selection mechanism to reduce computational cost by choosing only the crucial and non-repetitive patches for feature extraction. Additionally, we present a novel network structure that generates and utilizes pseudo frame global context to address the issue of incomplete views resulting from sparse inputs. By incorporating these new designs, our backbone can achieve both high performance and low computational cost. Extensive experiments on multiple datasets show that our approach reduces the computational cost by 74\% compared to ViT-B and 28\% compared to ResNet50, while the accuracy is on par with ViT-B and outperforms ResNet50 significantly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。