提出混合模态行人重识别新范式,解决跨模态混淆问题。
Mix-Modality Person Re-Identification: A New and Practical Paradigm
- 设计混合模态检索新范式,模拟真实场景多模态混合数据。
- 引入CIDHL损失函数,优化特征分布减少模态混淆。
- 适合关注真实应用中跨模态行人识别的科研与工程人员。
当前可见光-红外跨模态行人重识别研究主要集中在双模态互检范式,本文提出一种更贴近实际应用的混合模态检索新范式。现有方法在双模态互检中表现良好,但在新混合模态场景下因模态混淆导致性能显著下降。为此,本文定义了混合模态行人重识别(MM-ReID)任务,研究模态混合比例对性能的影响,并基于现有数据集构建符合新范式的测试集。为解决模态混淆问题,提出交叉身份判别调和损失(CIDHL),在超球面特征空间中拉近同身份样本中心,推开异身份样本中心,同时聚合同模态同身份样本。此外,提出模态桥相似性优化策略(MBSOS),利用画廊中相似桥样本优化查询与检索样本间的跨模态相似度。大量实验表明,相比原有跨模态方法在MM-ReID上的表现,加入CIDHL与MBSOS后实现普遍提升。
原文摘要 · Abstract (English)
Current visible-infrared cross-modality person re-identification research has only focused on exploring the bi-modality mutual retrieval paradigm, and we propose a new and more practical mix-modality retrieval paradigm. Existing Visible-Infrared person re-identification (VI-ReID) methods have achieved some results in the bi-modality mutual retrieval paradigm by learning the correspondence between visible and infrared modalities. However, significant performance degradation occurs due to the modality confusion problem when these methods are applied to the new mix-modality paradigm. Therefore, this paper proposes a Mix-Modality person re-identification (MM-ReID) task, explores the influence of modality mixing ratio on performance, and constructs mix-modality test sets for existing datasets according to the new mix-modality testing paradigm. To solve the modality confusion problem in MM-ReID, we propose a Cross-Identity Discrimination Harmonization Loss (CIDHL) adjusting the distribution of samples in the hyperspherical feature space, pulling the centers of samples with the same identity closer, and pushing away the centers of samples with different identities while aggregating samples with the same modality and the same identity. Furthermore, we propose a Modality Bridge Similarity Optimization Strategy (MBSOS) to optimize the cross-modality similarity between the query and queried samples with the help of the similar bridge sample in the gallery. Extensive experiments demonstrate that compared to the original performance of existing cross-modality methods on MM-ReID, the addition of our CIDHL and MBSOS demonstrates a general improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。