arXiv:2412.10785cs.CV2024-12被引 4

用风格潜空间扩散模型,精准生成有血缘关系的多样人脸。

StyleDiT: A Unified Framework for Diverse Child and Partner Faces Synthesis with Style Latent Diffusion Transformer

  • 结合StyleGAN与扩散模型,实现细粒度属性控制。
  • 在5个数据集上提升多样性与保真度,优于现有方法。
  • 支持父母与子女/配偶间互推,适合人脸生成研究者。

由于可用血缘人脸数据稀缺且质量低,血缘人脸合成面临挑战。现有方法难以在保持高多样性和保真度的同时精确控制年龄、性别等面部属性。为此,我们提出风格潜空间扩散变换器(StyleDiT),融合StyleGAN的丰富人脸先验与扩散模型对复杂血缘关系分布的建模能力,实现高质量、多样化的血缘人脸生成。在该框架中,风格潜空间提供细粒度属性控制,条件扩散模型则根据输入图像的血缘关系采样对应的StyleGAN潜变量,再由StyleGAN完成最终人脸解码。我们进一步引入关系特征引导(RTG)机制,可独立调控父母图像的影响,实现合成人脸保真度与多样性的精细调节。此外,该框架首次扩展至新领域:仅凭一个孩子的图像和一位家长的图像,预测其配偶的面部形象。大量实验表明,StyleDiT在多个数据集上显著优于现有方法,在多样性与保真度之间取得优异平衡。

原文摘要 · Abstract (English)

Kinship face synthesis is a challenging problem due to the scarcity and low quality of the available kinship data. Existing methods often struggle to generate descendants with both high diversity and fidelity while precisely controlling facial attributes such as age and gender. To address these issues, we propose the Style Latent Diffusion Transformer (StyleDiT), a novel framework that integrates the strengths of StyleGAN with the diffusion model to generate high-quality and diverse kinship faces. In this framework, the rich facial priors of StyleGAN enable fine-grained attribute control, while our conditional diffusion model is used to sample a StyleGAN latent aligned with the kinship relationship of conditioning images by utilizing the advantage of modeling complex kinship relationship distribution. StyleGAN then handles latent decoding for final face generation. Additionally, we introduce the Relational Trait Guidance (RTG) mechanism, enabling independent control of influencing conditions, such as each parent's facial image. RTG also enables a fine-grained adjustment between the diversity and fidelity in synthesized faces. Furthermore, we extend the application to an unexplored domain: predicting a partner's facial images using a child's image and one parent's image within the same framework. Extensive experiments demonstrate that our StyleDiT outperforms existing methods by striking an excellent balance between generating diverse and high-fidelity kinship faces.

人脸生成扩散模型血缘关系风格控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。