arXiv:2506.08953cs.CV2025-06CVPR

用视觉变压器解决可见光与红外人体识别,仅编码相机信息就达顶尖效果。

Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF

  • 引入侧信息嵌入,仅编码相机参数提升跨谱匹配性能。
  • 在LLCM数据集上达到当前最优,验证相机信息的关键作用。
  • 首次分析跨距离红外遮挡问题,适合多模态识别研究者。

视觉变压器(ViT)在人脸识别、人体识别等生物特征任务中表现优异。本文将预训练于可见光(VIS)图像的ViT模型迁移至跨谱人体识别任务,即匹配可见光与红外(IR)域图像。近期ViT架构探索了除传统位置嵌入外的额外嵌入方式。基于此,我们引入侧信息嵌入(SIE),编码域和相机信息以增强跨谱匹配。令人意外的是,仅编码相机信息而无需显式包含域信息,便在LLCM数据集上取得当前最优性能。尽管可见光人体重识别中的遮挡问题已广泛研究,但可见-红外(VI)重识别中的遮挡仍基本未被探索,主要因现有数据集(如LLCM、SYSU-MM01、RegDB)多为完整无遮挡人体图像。为填补该空白,我们利用IARPA Janus Benchmark Multi-Domain Face(IJB-MDF)数据集,分析不同距离下引起的遮挡影响,该数据集提供多种距离拍摄的可见光与红外图像,支持跨距离、跨谱评估。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) have demonstrated impressive performance across a wide range of biometric tasks, including face and body recognition. In this work, we adapt a ViT model pretrained on visible (VIS) imagery to the challenging problem of cross-spectral body recognition, which involves matching images captured in the visible and infrared (IR) domains. Recent ViT architectures have explored incorporating additional embeddings beyond traditional positional embeddings. Building on this idea, we integrate Side Information Embedding (SIE) and examine the impact of encoding domain and camera information to enhance cross-spectral matching. Surprisingly, our results show that encoding only camera information - without explicitly incorporating domain information - achieves state-of-the-art performance on the LLCM dataset. While occlusion handling has been extensively studied in visible-spectrum person re-identification (Re-ID), occlusions in visible-infrared (VI) Re-ID remain largely underexplored - primarily because existing VI-ReID datasets, such as LLCM, SYSU-MM01, and RegDB, predominantly feature full-body, unoccluded images. To address this gap, we analyze the impact of range-induced occlusions using the IARPA Janus Benchmark Multi-Domain Face (IJB-MDF) dataset, which provides a diverse set of visible and infrared images captured at various distances, enabling cross-range, cross-spectral evaluations.

跨谱识别视觉变压器侧信息嵌入遮挡分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。