用CLIP提升单摄像头行人重识别,让模型无视摄像头差异
CLIP-based Camera-Agnostic Feature Learning for Intra-camera Person Re-Identification

- 设计双模块引导模型学习跨摄像头通用行人特征
- 在MSMT17上达58.9% mAP,领先当前方法7.6%
- 适合需要跨镜头识别人的安防与监控场景
对比语言-图像预训练(CLIP)模型在传统行人重识别任务中表现优异,因其能为行人图像生成文本描述。然而,直接应用于单摄像头监督行人重识别(ICS ReID)存在挑战:该任务需在每个摄像头内独立标注身份,无法跨摄像头建立关联,限制了基于文本的增强效果。为此,我们提出一种新框架——基于CLIP的相机无关特征学习(CCAFL),通过两个定制模块引导模型主动学习相机无关的行人特征:摄像头内判别学习(ICDL)和摄像头间对抗学习(ICAL)。首先,为每个摄像头内的行人图像构建可学习的文本提示,获取后续学习所需的语义监督信号;其次,设计ICDL通过考虑每个摄像头内的难正例与难负例,增强类间差异,学习更细粒度的摄像头内特征;此外,提出ICAL通过惩罚模型预测图像来源摄像头的能力,减少不同摄像头间的特征差异,提升模型对多视角行人的识别能力。在多个主流ReID数据集上的实验验证了该方法的有效性。尤其在具有挑战性的MSMT17数据集上,mAP达到58.9%,优于现有最先进方法7.6%。代码将开源。
原文摘要 · Abstract (English)
Contrastive Language-Image Pre-Training (CLIP) model excels in traditional person re-identification (ReID) tasks due to its inherent advantage in generating textual descriptions for pedestrian images. However, applying CLIP directly to intra-camera supervised person re-identification (ICS ReID) presents challenges. ICS ReID requires independent identity labeling within each camera, without associations across cameras. This limits the effectiveness of text-based enhancements. To address this, we propose a novel framework called CLIP-based Camera-Agnostic Feature Learning (CCAFL) for ICS ReID. Accordingly, two custom modules are designed to guide the model to actively learn camera-agnostic pedestrian features: Intra-Camera Discriminative Learning (ICDL) and Inter-Camera Adversarial Learning (ICAL). Specifically, we first establish learnable textual prompts for intra-camera pedestrian images to obtain crucial semantic supervision signals for subsequent intra- and inter-camera learning. Then, we design ICDL to increase inter-class variation by considering the hard positive and hard negative samples within each camera, thereby learning intra-camera finer-grained pedestrian features. Additionally, we propose ICAL to reduce inter-camera pedestrian feature discrepancies by penalizing the model's ability to predict the camera from which a pedestrian image originates, thus enhancing the model's capability to recognize pedestrians from different viewpoints. Extensive experiments on popular ReID datasets demonstrate the effectiveness of our approach. Especially, on the challenging MSMT17 dataset, we arrive at 58.9\% in terms of mAP accuracy, surpassing state-of-the-art methods by 7.6\%. Code will be available at: https://github.com/Trangle12/CCAFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。