用3D约束提升合成人脸风格迁移一致性。
TCDiff: Triple Condition Diffusion Model with 3D Constraints for Stylizing Synthetic Faces
- 引入2D与3D双重约束，增强生成人脸身份一致性。
- 在1k/2k/5k类数据上优于现有合成数据集。
- 适合需要高保真且身份一致的合成人脸研究者。
稳健的人脸识别模型需在包含大量被试及每被试多样本、涵盖姿态、表情、年龄、噪声和遮挡等多样条件的数据集上训练。由于伦理与隐私问题，大规模真实人脸数据集（如MS1MV3）已停止使用，因此提出基于GAN和扩散模型的合成人脸生成方法（如SYNFace、SFace、DigiFace-1M、IDiff-Face、DCFace、GANDiffFace），以满足需求。部分方法可生成高保真度真实人脸，但类内差异低；另一些则生成类内差异高，但身份一致性差。本文提出三重条件扩散模型（TCDiff），通过2D与3D面部约束，提升从真实到合成人脸的风格迁移效果，在保持必要类内差异的同时增强身份一致性。使用新数据集分别以1000、2000、5000类进行训练，在LFW、CFP-FP、AgeDB和BUPT等真实人脸基准测试中表现超越当前最优合成数据集。源代码已公开于：https://github.com/BOVIFOCR/tcdiff。
原文摘要 · Abstract (English)
A robust face recognition model must be trained using datasets that include a large number of subjects and numerous samples per subject under varying conditions (such as pose, expression, age, noise, and occlusion). Due to ethical and privacy concerns, large-scale real face datasets have been discontinued, such as MS1MV3, and synthetic face generators have been proposed, utilizing GANs and Diffusion Models, such as SYNFace, SFace, DigiFace-1M, IDiff-Face, DCFace, and GANDiffFace, aiming to supply this demand. Some of these methods can produce high-fidelity realistic faces, but with low intra-class variance, while others generate high-variance faces with low identity consistency. In this paper, we propose a Triple Condition Diffusion Model (TCDiff) to improve face style transfer from real to synthetic faces through 2D and 3D facial constraints, enhancing face identity consistency while keeping the necessary high intra-class variance. Face recognition experiments using 1k, 2k, and 5k classes of our new dataset for training outperform state-of-the-art synthetic datasets in real face benchmarks such as LFW, CFP-FP, AgeDB, and BUPT. Our source code is available at: https://github.com/BOVIFOCR/tcdiff.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。