通过多频段专家网络提升可见光与红外行人重识别性能
MFEN:Multi-Frequency Expert Network for Visible-Infrared Person Re-ID

- 设计多频段专家网络,自适应融合不同频率成分
- 在三个数据集上达到领先效果,显著降低模态差异影响
- 适合需要跨模态图像匹配的安防与智能监控场景
可见光-红外行人重识别(VI-ReID)因两种图像间模态差异大而具有挑战性。我们认为这种差异主要源于光照条件的不同,包括光波长和光源类型差异。近期基于频率的方法取得显著进展,因频率信息能更好提取与身份相关的轮廓和细节,同时排除无关的光照和颜色信息。然而,现有方法或未区分不同频率带,或仅关注单一频带,在多样光照条件下表现不足。为此,我们提出多频段专家网络(MFEN),通过混合专家结构实现多频段调制并自适应融合不同频带。此外,引入随机频域增强(RFA)和频域辅助优化(FAO)以更好训练MFEN。三模块互补,共同捕捉关键频域细节,实现鲁棒表征学习。在三个VI-ReID数据集上的大量实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Visible-infrared person re-identification (VI-ReID) is challenging due to the large modality discrepancy between visible and infrared images. We contend that this discrepancy is largely related to differing lighting conditions, including differences in light wavelength and light source type. Recently, frequency-based VI-ReID approaches have achieved notable success because frequency information can better extract identity-relevant contours and details while excluding irrelevant lighting and color. However, existing methods either do not distinguish different frequency bands or focus on only one band, which is insufficient under diverse lighting conditions. To perform comprehensive frequency domain learning, we propose a Multi-Frequency Expert Network (MFEN) that enables multi-frequency modulation and adaptively combines different bands through a mixture-of-experts design. We further introduce Random Frequency Augmentation (RFA) and Frequency Auxiliary Optimization (FAO) to better train MFEN. The three modules are complementary and jointly capture critical frequency-domain details for robust representation learning. Extensive experiments on three VI-ReID datasets demonstrate the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。