arXiv:2504.14847cs.CV2025-04被引 4

用图推理提升多模态重识别,自动过滤低质特征并补全缺失信息。

Reliable Multi-Modal Object Re-Identification via Modality-Aware Graph Reasoning

  • 构建模态感知图,捕捉局部块间关系以提取细粒度特征。
  • 通过选择性节点交换缓解低质量特征干扰,提升判别性表示。
  • 能重建缺失模态信息,适合跨模态数据不全的场景。

多模态数据包含丰富且多样化的物体信息,对有效模态交互至关重要。然而,现有方法常忽略局部特征的质量差异,未能充分利用模态间的互补信息,尤其在低质量特征情况下表现不佳。本文提出一种新颖的图推理模型——模态感知图推理网络(MGRNet)。首先,构建模态感知图以增强细粒度局部细节的提取,有效建模块间关系;随后,采用选择性图节点交换操作,结合局部与全局信息,缓解低质量局部特征的负面影响,强化判别性表征;最后,将交换后的模态感知图输入局部感知图推理模块,传播多模态信息,生成可靠特征表示。该方法还能利用内在结构关系重建缺失模态信息,降低不同模态间的差异。在四个基准数据集(RGBNT201、Market1501-MM、RGBNT100、MSVR310)上的实验表明,该方法在多模态物体重识别任务中达到领先性能。代码将在录用后公开。

原文摘要 · Abstract (English)

Multi-modal data provides abundant and diverse object information, crucial for effective modal interactions in Re-Identification (ReID) tasks. However, existing approaches often overlook the quality variations in local features and fail to fully leverage the complementary information across modalities, particularly in the case of low-quality features. In this paper, we propose to address this issue by leveraging a novel graph reasoning model, termed the Modality-aware Graph Reasoning Network (MGRNet). Specifically, we first construct modality-aware graphs to enhance the extraction of fine-grained local details by effectively capturing and modeling the relationships between patches. Subsequently, the selective graph nodes swap operation is employed to alleviate the adverse effects of low-quality local features by considering both local and global information, enhancing the representation of discriminative information. Finally, the swapped modality-aware graphs are fed into the local-aware graph reasoning module, which propagates multi-modal information to yield a reliable feature representation. Another advantage of the proposed graph reasoning approach is its ability to reconstruct missing modal information by exploiting inherent structural relationships, thereby minimizing disparities between different modalities. Experimental results on four benchmarks (RGBNT201, Market1501-MM, RGBNT100, MSVR310) indicate that the proposed method achieves state-of-the-art performance in multi-modal object ReID. The code for our method will be available upon acceptance.

多模态重识别图神经网络特征融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。