用扩散模型生成跨模态人像数据，提升红外可见光行人重识别性能。
Diffusion-based Synthetic Data Generation for Visible-Infrared Person Re-Identification
- 将身份与模态解耦，用扩散模型生成保持身份一致的红外-可见光图像对。
- 在LLCM数据集上，训练模型mAP提升约9%，超越基线。
- 适合需要隐私保护、数据稀缺场景下的行人重识别研究者。
模型性能与训练数据量密切相关。在可见光-红外行人重识别（VI-ReID）任务中，收集并标注大规模跨摄像头、跨模态的个体图像耗时费力、成本高昂，且需遵守数据保护法规，难以满足数据需求。现有研究尝试生成合成数据作为真实数据采集的高效替代方案，但针对VI-ReID的专用数据生成方法尚缺。本文提出一种名为DiVE（Diffusion-based VI-ReID data Expansion）的新框架，通过解耦身份与模态，自动生成大量保持身份一致的RGB-IR配对图像，以提升VI-ReID模型性能。具体地，身份表征从同身份样本中提取，而模态信息通过在特定模态数据上微调Stable Diffusion（SD）获得。该方法将文本驱动图像生成扩展为身份保持的多模态图像生成，显著降低数据采集与标注成本，并可直接用于重识别模型训练。实验表明，使用DiVE生成的合成数据训练的模型表现显著提升。尤其在LLCM数据集上，最先进方法CAJ在合成数据训练下，mAP相较基线提升约9%。代码已开源：https://github.com/BorgDiven/DiVE。
原文摘要 · Abstract (English)
The performance of models is intricately linked to the abundance of training data. In Visible-Infrared person Re-IDentification (VI-ReID) tasks, collecting and annotating large-scale images of each individual under various cameras and modalities is tedious, time-expensive, costly and must comply with data protection laws, posing a severe challenge in meeting dataset requirements. Current research investigates the generation of synthetic data as an efficient and privacy-ensuring alternative to collecting real data in the field. However, a specific data synthesis technique tailored for VI-ReID models has yet to be explored. In this paper, we present a novel data generation framework, dubbed Diffusion-based VI-ReID data Expansion (DiVE), that automatically obtain massive RGB-IR paired images with identity preserving by decoupling identity and modality to improve the performance of VI-ReID models. Specifically, identity representation is acquired from a set of samples sharing the same ID, whereas the modality of images is learned by fine-tuning the Stable Diffusion (SD) on modality-specific data. DiVE extend the text-driven image synthesis to identity-preserving RGB-IR multimodal image synthesis. This approach significantly reduces data collection and annotation costs by directly incorporating synthetic data into ReID model training. Experiments have demonstrated that VI-ReID models trained on synthetic data produced by DiVE consistently exhibit notable enhancements. In particular, the state-of-the-art method, CAJ, trained with synthetic images, achieves an improvement of about $9\%$ in mAP over the baseline on the LLCM dataset. Code: https://github.com/BorgDiven/DiVE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。