用3D模型指导扩散生成人脸，实现身份一致且风格多样的数据合成。
Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion
- 基于3DMM提取精细风格特征，结合扩散模型生成人脸。
- 在多种真实风格分布下采样，提升同一人不同风格的多样性。
- 适合需要高质量人脸数据训练的识别模型研究者使用。
身份保持的人脸合成旨在生成可替代真实数据用于训练人脸识别模型的虚拟人脸图像。尽管先前方法试图在保持身份一致性的同时实现风格多样化，但仍面临二者间的权衡。本文指出其将风格变化视为与主体无关的缺陷，并观察到现实中个体具有独特的、与自身相关的风格特征。为此提出MorphFace，一种基于扩散模型的人脸生成器，从3D可变形模型（3DMM）的渲染结果中学习细粒度的面部风格（如形状、姿态、表情），同时从现成的识别模型中学习身份信息。生成时，该模型以未标注的合成人脸的新身份和从真实世界先验分布中统计采样的新风格为条件，特别考虑了个体内部变异性和个体独特性。采用上下文融合策略增强对身份与风格条件的响应能力。大量实验表明，MorphFace在人脸识别效果上优于现有最佳方法。
原文摘要 · Abstract (English)
Identity-preserving face synthesis aims to generate synthetic face images of virtual subjects that can substitute real-world data for training face recognition models. While prior arts strive to create images with consistent identities and diverse styles, they face a trade-off between them. Identifying their limitation of treating style variation as subject-agnostic and observing that real-world persons actually have distinct, subject-specific styles, this paper introduces MorphFace, a diffusion-based face generator. The generator learns fine-grained facial styles, e.g., shape, pose and expression, from the renderings of a 3D morphable model (3DMM). It also learns identities from an off-the-shelf recognition model. To create virtual faces, the generator is conditioned on novel identities of unlabeled synthetic faces, and novel styles that are statistically sampled from a real-world prior distribution. The sampling especially accounts for both intra-subject variation and subject distinctiveness. A context blending strategy is employed to enhance the generator's responsiveness to identity and style conditions. Extensive experiments show that MorphFace outperforms the best prior arts in face recognition efficacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。