用原型驱动生成多特征,提升可见光与红外行人重识别效果
Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification
- 通过可学习原型挖掘跨模态语义相似性
- 在SYSU-MM01和LLCM上达到当前最好性能
- 适合解决红外与可见光图像差异大的场景
可见-红外行人重识别的主要挑战源于可见(vis)与红外(ir)图像之间的模态间和模态内差异,且受视角变化和不规则运动影响。现有方法常采用水平分割对齐局部特征,但易引入误差且对模态差异缓解有限。本文提出原型驱动的多特征生成框架(PDM),通过构建多样化特征并挖掘潜在语义相似性以实现模态对齐。PDM包含两个核心组件:多特征生成模块(MFGM)和原型学习模块(PLM)。MFGM生成围绕共享特征分布的多样性特征以表征行人;PLM利用可学习原型挖掘可见与红外局部特征间的潜在语义关联,促进实例级跨模态对齐。引入余弦异质性损失增强原型多样性,以提取更丰富的局部特征。在SYSU-MM01和LLCM数据集上的大量实验表明,本方法达到最优性能。代码已开源。
原文摘要 · Abstract (English)
The primary challenges in visible-infrared person re-identification arise from the differences between visible (vis) and infrared (ir) images, including inter-modal and intra-modal variations. These challenges are further complicated by varying viewpoints and irregular movements. Existing methods often rely on horizontal partitioning to align part-level features, which can introduce inaccuracies and have limited effectiveness in reducing modality discrepancies. In this paper, we propose a novel Prototype-Driven Multi-feature generation framework (PDM) aimed at mitigating cross-modal discrepancies by constructing diversified features and mining latent semantically similar features for modal alignment. PDM comprises two key components: Multi-Feature Generation Module (MFGM) and Prototype Learning Module (PLM). The MFGM generates diversity features closely distributed from modality-shared features to represent pedestrians. Additionally, the PLM utilizes learnable prototypes to excavate latent semantic similarities among local features between visible and infrared modalities, thereby facilitating cross-modal instance-level alignment. We introduce the cosine heterogeneity loss to enhance prototype diversity for extracting rich local features. Extensive experiments conducted on the SYSU-MM01 and LLCM datasets demonstrate that our approach achieves state-of-the-art performance. Our codes are available at https://github.com/mmunhappy/ICASSP2025-PDM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。