用扩散模型生成换装图像,提升人物重识别精度
DLCR: A Generative Data Expansion Framework via Diffusion for Clothes-Changing Person Re-ID
- 结合扩散模型与大语言模型,精准生成不同穿搭的人像
- 在5个数据集上生成超210万张图像,服装多样性提升10倍
- 适用于需要强身份鲁棒性的跨摄像头重识别任务
随着生成式扩散模型的兴起,一个开放性问题在于:这些模型生成的图像能否用于学习更优的视觉表征。尽管生成数据扩展对简单视觉任务有效,本文探索其在更具挑战性的判别任务——换装人物重识别(CC-ReID)中的效果。该任务旨在匹配跨摄像头出现且更换衣物的人物。现有模型受限于数据集中服装多样性不足,而如何生成保留关键身份特征的新增数据仍是难题。为此,我们提出DLCR框架,利用预训练扩散模型与大语言模型(LLMs),根据文本提示生成多样化的个体穿搭图像。我们在五个基准数据集(PRCC、CCVID、LaST、VC-Clothes、LTCC)上生成额外数据,使服装多样性提升10倍,共生成超过210万张图像。DLCR采用基于文本引导的图像修复机制,在仅修改衣物的同时保留个人可识别特征。借助海量数据,我们引入两种新策略——渐进式学习与测试时预测优化,分别降低训练时间并进一步提升性能。在PRCC数据集上,使用DLCR生成数据训练的CAL模型达到11.3%的top-1准确率提升。代码与生成数据已公开:https://github.com/CroitoruAlin/dlcr。
原文摘要 · Abstract (English)
With the recent exhibited strength of generative diffusion models, an open research question is if images generated by these models can be used to learn better visual representations. While this generative data expansion may suffice for easier visual tasks, we explore its efficacy on a more difficult discriminative task: clothes-changing person re-identification (CC-ReID). CC-ReID aims to match people appearing in non-overlapping cameras, even when they change their clothes across cameras. Not only are current CC-ReID models constrained by the limited diversity of clothing in current CC-ReID datasets, but generating additional data that retains important personal features for accurate identification is a current challenge. To address this issue we propose DLCR, a novel data expansion framework that leverages pre-trained diffusion and large language models (LLMs) to accurately generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets (PRCC, CCVID, LaST, VC-Clothes, and LTCC) and increase their clothing diversity by 10X, totaling over 2.1M images generated. DLCR employs diffusion-based text-guided inpainting, conditioned on clothing prompts constructed using LLMs, to generate synthetic data that only modifies a subject's clothes while preserving their personally identifiable features. With this massive increase in data, we introduce two novel strategies - progressive learning and test-time prediction refinement - that respectively reduce training time and further boosts CC-ReID performance. On the PRCC dataset, we obtain a large top-1 accuracy improvement of 11.3% by training CAL, a previous state of the art (SOTA) method, with DLCR-generated data. We publicly release our code and generated data for each dataset here: https://github.com/CroitoruAlin/dlcr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。