用文本提示引导特征解耦,提升遮挡下行人重识别准确率
ProFD: Prompt-Guided Feature Disentangling for Occluded Person Re-Identification

- 通过特定文本提示让模型具备空间感知,对齐视觉与语义特征
- 采用混合注意力解码器,降低外部分割噪声影响,保持定位精准
- 结合自蒸馏策略避免遗忘预训练知识,适合遮挡场景下的行人识别
为解决行人重识别中遮挡带来的挑战,现有方法常依赖外部空间信息提取部件特征,但因遮挡导致外观信息缺失及外部模型引入的噪声,难以在有限数据下准确学习人体部件特征并精确定位,最终造成特征错位。为此,本文提出提示引导的特征解耦方法(ProFD),利用文本模态中丰富的预训练知识,帮助模型生成对齐良好的部件特征。ProFD首先设计部件级提示,并结合噪声分割掩码初步对齐视觉与文本嵌入,使文本提示具备空间意识;其次,采用混合注意力解码器,在解码过程中保证空间与语义一致性,有效抑制噪声干扰;此外,通过自蒸馏策略保留CLIP的预训练知识,防止灾难性遗忘。在Market1501、DukeMTMC-ReID、Occluded-Duke、Occluded-ReID和P-DukeMTMC数据集上的实验表明,ProFD达到当前最优性能。
原文摘要 · Abstract (English)
To address the occlusion issues in person Re-Identification (ReID) tasks, many methods have been proposed to extract part features by introducing external spatial information. However, due to missing part appearance information caused by occlusion and noisy spatial information from external model, these purely vision-based approaches fail to correctly learn the features of human body parts from limited training data and struggle in accurately locating body parts, ultimately leading to misaligned part features. To tackle these challenges, we propose a Prompt-guided Feature Disentangling method (ProFD), which leverages the rich pre-trained knowledge in the textual modality facilitate model to generate well-aligned part features. ProFD first designs part-specific prompts and utilizes noisy segmentation mask to preliminarily align visual and textual embedding, enabling the textual prompts to have spatial awareness. Furthermore, to alleviate the noise from external masks, ProFD adopts a hybrid-attention decoder, ensuring spatial and semantic consistency during the decoding process to minimize noise impact. Additionally, to avoid catastrophic forgetting, we employ a self-distillation strategy, retaining pre-trained knowledge of CLIP to mitigate over-fitting. Evaluation results on the Market1501, DukeMTMC-ReID, Occluded-Duke, Occluded-ReID, and P-DukeMTMC datasets demonstrate that ProFD achieves state-of-the-art results. Our project is available at: https://github.com/Cuixxx/ProFD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。