arXiv:2503.23722cs.CV2025-03被引 10

用文本属性知识提升航拍与地面视角行人重识别效果

LATex: Leveraging Attribute-based Text Knowledge for Aerial-Ground Person Re-Identification

  • 通过提示调优利用文本属性信息增强特征表达
  • 在三个基准上达到新最好结果,最高提升6.2%
  • 适合关注跨视角行人识别与少样本学习的研究者

作为智能交通系统的重要任务,航拍-地面行人重识别(AG-ReID)旨在不同视角的异构摄像头间检索特定人员。以往方法多采用深度学习模型提取视角不变特征,但常忽略人物属性的语义信息。同时,现有训练策略依赖大规模模型的全量微调,显著增加训练成本。为此,本文提出LATex框架,采用提示调优策略,充分挖掘基于属性的文本知识。具体而言,基于对比语言-图像预训练(CLIP)模型,首先设计属性感知图像编码器(AIE),从输入图像中提取全局语义特征和属性感知特征;接着,构建提示属性分类器组(PACG)预测人物属性并生成属性表示;最后,设计耦合提示模板(CPT),将属性表示与视图信息转化为结构化句子,经由CLIP文本编码器处理后生成更具判别性的特征。实验表明,该框架在三个AG-ReID基准上均取得显著性能提升,最优结果较基线提高6.2%。源代码已开源。

原文摘要 · Abstract (English)

As an important task in intelligent transportation systems, Aerial-Ground person Re-IDentification (AG-ReID) aims to retrieve specific persons across heterogeneous cameras in different viewpoints. Previous methods typically adopt deep learning-based models, focusing on extracting view-invariant features. However, they usually overlook the semantic information in person attributes. In addition, existing training strategies often rely on full fine-tuning large-scale models, which significantly increases training costs. To address these issues, we propose a novel framework named LATex for AG-ReID, which adopts prompt-tuning strategies to leverage attribute-based text knowledge. Specifically, with the Contrastive Language-Image Pre-training (CLIP) model, we first propose an Attribute-aware Image Encoder (AIE) to extract both global semantic features and attribute-aware features from input images. Then, with these features, we propose a Prompted Attribute Classifier Group (PACG) to predict person attributes and obtain attribute representations. Finally, we design a Coupled Prompt Template (CPT) to transform attribute representations and view information into structured sentences. These sentences are processed by the text encoder of CLIP to generate more discriminative features. As a result, our framework can fully leverage attribute-based text knowledge to improve AG-ReID performance. Extensive experiments on three AG-ReID benchmarks demonstrate the effectiveness of our proposed methods. The source code is available at https://github.com/kevinhu314/LATex.

行人重识别视觉语言模型提示调优

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。