提出可任意模态间匹配的行人重识别框架,解决跨传感器识别难题。
MDReID: Modality-Decoupled Learning for Any-to-Any Multi-Modal Object Re-Identification
- 将模态特征解耦为共享与特定两部分,提升跨模态适应性
- 在三种数据集上实现最高11.5%的准确率提升,跨模态平均增益超10%
- 适合真实场景中多传感器协同的行人识别系统应用
现实世界中的行人重识别系统常面临模态不一致问题,即查询与库图像来自不同传感器(如可见光、近红外、热成像)。然而,现有方法大多假设模态匹配,限制了实际应用中的鲁棒性与可扩展性。为此,本文提出MDReID,一种灵活的任意模态间图像级重识别框架,适用于模态匹配与不匹配两种场景。其核心思想是将模态信息分解为可预测且可迁移的共享特征和捕捉独特模态特性的特定特征。为此,引入两个关键组件:模态解耦学习(MDL)与模态感知度量学习(MML)。MDL显式分解模态特征,实现跨模态有效检索;MML通过强制两部分正交互补,增强跨模态判别能力。在三个挑战性多模态重识别基准(RGBNT201、RGBNT100、MSVR310)上的大量实验表明,MDReID表现优异:在一般模态匹配场景下,分别取得9.8%、3.0%、11.5%的mAP提升;在模态不匹配场景下,平均提升分别为3.4%、11.8%、10.9%。代码已开源。
原文摘要 · Abstract (English)
Real-world object re-identification (ReID) systems often face modality inconsistencies, where query and gallery images come from different sensors (e.g., RGB, NIR, TIR). However, most existing methods assume modality-matched conditions, which limits their robustness and scalability in practical applications. To address this challenge, we propose MDReID, a flexible any-to-any image-level ReID framework designed to operate under both modality-matched and modality-mismatched scenarios. MDReID builds on the insight that modality information can be decomposed into two components: modality-shared features that are predictable and transferable, and modality-specific features that capture unique, modality-dependent characteristics. To effectively leverage this, MDReID introduces two key components: the Modality Decoupling Learning (MDL) and Modality-aware Metric Learning (MML). Specifically, MDL explicitly decomposes modality features into modality-shared and modality-specific representations, enabling effective retrieval in both modality-aligned and mismatched scenarios. MML, a tailored metric learning strategy, further enforces orthogonality and complementarity between the two components to enhance discriminative power across modalities. Extensive experiments conducted on three challenging multi-modality ReID benchmarks (RGBNT201, RGBNT100, MSVR310) consistently demonstrate the superiority of MDReID. Notably, MDReID achieves significant mAP improvements of 9.8\%, 3.0\%, and 11.5\% in general modality-matched scenarios, and average gains of 3.4\%, 11.8\%, and 10.9\% in modality-mismatched scenarios, respectively. The code is available at: \textcolor{magenta}{https://github.com/stone96123/MDReID}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。