用虚拟数据训练更私密的行人重识别模型,效果超越真实与现有虚拟数据集。
A Dual-stage Prompt-driven Privacy-preserving Paradigm for Person Re-Identification
- 分两阶段生成虚拟图像:先用多维提示词驱动扩散模型合成13万张图
- 通过文本反演分离风格与内容,学习跨域不变特征,提升泛化能力
- 适合关注隐私保护与跨域性能的行人重识别研究者
随着数据隐私问题日益受到关注,研究人员开始使用虚拟数据替代敏感的真实图像来训练行人重识别(Re-ID)模型。然而,现有的游戏引擎生成的虚拟数据集仍面临构建复杂和域泛化能力差的问题,难以在实际场景中应用。为此,我们提出双阶段提示驱动隐私保护范式(DPPP)。第一阶段中,利用包含行人外观、光照、视角等多维度属性的丰富提示词,驱动扩散模型端到端生成多样化数据,构建大规模虚拟数据集GenePerson,包含130,519张图像,覆盖6,641个身份。第二阶段提出提示驱动解耦机制(PDM),借助对比学习,使用两个文本反演网络将图像映射为表示风格和内容的伪词,从而构建风格解耦的内容提示,指导模型在图像层面学习域不变的内容特征。实验表明,基于GenePerson并结合PDM训练的模型在泛化性能上达到当前最优,优于主流真实和虚拟Re-ID数据集上的表现。
原文摘要 · Abstract (English)
With growing concerns over data privacy, researchers have started using virtual data as an alternative to sensitive real-world images for training person re-identification (Re-ID) models. However, existing virtual datasets produced by game engines still face challenges such as complex construction and poor domain generalization, making them difficult to apply in real scenarios. To address these challenges, we propose a Dual-stage Prompt-driven Privacy-preserving Paradigm (DPPP). In the first stage, we generate rich prompts incorporating multi-dimensional attributes such as pedestrian appearance, illumination, and viewpoint that drive the diffusion model to synthesize diverse data end-to-end, building a large-scale virtual dataset named GenePerson with 130,519 images of 6,641 identities. In the second stage, we propose a Prompt-driven Disentanglement Mechanism (PDM) to learn domain-invariant generalization features. With the aid of contrastive learning, we employ two textual inversion networks to map images into pseudo-words representing style and content, respectively, thereby constructing style-disentangled content prompts to guide the model in learning domain-invariant content features at the image level. Experiments demonstrate that models trained on GenePerson with PDM achieve state-of-the-art generalization performance, surpassing those on popular real and virtual Re-ID datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。