混合生成融合方法高效构建无重叠人脸数据集,获ICCV挑战赛第一名。
Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation
- 结合聚类与GPT-4o验证清洗原始数据,保留最大一致身份簇。
- 用扩散模型生成单参考图,再通过Vec2Face快速扩展至49张,每身份50图。
- 采用课程学习策略缓解合成身份视觉相似问题,适合大规模人脸识别训练。
本文介绍参加DataCV ICCV挑战赛的方法,旨在构建高质量人脸数据集以训练人脸识别模型。该数据集必须不包含与现有公开人脸数据集重叠的身份。我们首先对基础数据集HSFace进行彻底清洗,通过融合面部嵌入聚类与GPT-4o辅助验证的专家混合(MoE)策略,识别并移除错误标注或不一致的身份。保留最大的一致身份簇,并将每个身份的数据增强至固定数量。为增加多样性,使用经过提示工程优化的Stable Diffusion生成合成身份,每身份仅生成一张参考图像,再利用Vec2Face快速生成49个身份一致的变体。该混合方法融合了GAN与扩散模型样本,实现高效、多样且高质量数据集的构建。为解决合成身份间视觉相似度过高的问题,采用课程学习策略,将其置于训练初期,使模型从简单样本逐步过渡到复杂样本。最终数据集每身份50张图像,所有新生成身份均经主流人脸数据集验证,确保无身份泄露。本方法在比赛中获得第一名,实验表明,该数据集在10K、20K和100K身份规模下均显著提升模型性能。代码已开源。
原文摘要 · Abstract (English)
In this paper, we present our approach to the DataCV ICCV Challenge, which centers on building a high-quality face dataset to train a face recognition model. The constructed dataset must not contain identities overlapping with any existing public face datasets. To handle this challenge, we begin with a thorough cleaning of the baseline HSFace dataset, identifying and removing mislabeled or inconsistent identities through a Mixture-of-Experts (MoE) strategy combining face embedding clustering and GPT-4o-assisted verification. We retain the largest consistent identity cluster and apply data augmentation up to a fixed number of images per identity. To further diversify the dataset, we generate synthetic identities using Stable Diffusion with prompt engineering. As diffusion models are computationally intensive, we generate only one reference image per identity and efficiently expand it using Vec2Face, which rapidly produces 49 identity-consistent variants. This hybrid approach fuses GAN-based and diffusion-based samples, enabling efficient construction of a diverse and high-quality dataset. To address the high visual similarity among synthetic identities, we adopt a curriculum learning strategy by placing them early in the training schedule, allowing the model to progress from easier to harder samples. Our final dataset contains 50 images per identity, and all newly generated identities are checked with mainstream face datasets to ensure no identity leakage. Our method achieves \textbf{1st place} in the competition, and experimental results show that our dataset improves model performance across 10K, 20K, and 100K identity scales. Code is available at https://github.com/Ferry-Li/datacv_fr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。