仅用一张照片生成通用3D人脸,支持各种真实场景下的人脸输入。
Single Image, Any Face: Generalisable 3D Face Generation
- 基于多视角一致的扩散模型,从单图生成多视角图像并构建神经表面。
- 无需3D真值监督,直接从输入图估计个性化的网格结构,实现跨域泛化。
- 首次建立跨领域单图3D人脸生成基准,适合需要真实世界人脸建模的场景。
从单张非约束图像生成3D人脸是视觉与图形应用中的基础任务。尽管生成模型已取得显著进展,现有方法或设计上不适用于人脸,或难以从受限训练域泛化到真实复杂图像。为此,我们提出Gen3D-Face模型,在多视角一致的扩散框架下,仅需单张输入图像即可生成3D人脸。给定输入图像,模型先生成多视角图像,再进行神经表面构建。为融合面部几何信息并保持对真实场景图像的泛化能力,我们直接从输入图估计个体化网格,实现无需真值3D监督的训练与评估。重要的是,我们引入多视角联合生成机制,提升不同视角间外观的一致性。据我们所知,这是首个在跨域通用人类主体上实现单图生成逼真3D人脸的尝试与基准。大量实验表明,该方法在跨域设置下优于先前方法,在同域设置下排名领先。
原文摘要 · Abstract (English)
The creation of 3D human face avatars from a single unconstrained image is a fundamental task that underlies numerous real-world vision and graphics applications. Despite the significant progress made in generative models, existing methods are either less suited in design for human faces or fail to generalise from the restrictive training domain to unconstrained facial images. To address these limitations, we propose a novel model, Gen3D-Face, which generates 3D human faces with unconstrained single image input within a multi-view consistent diffusion framework. Given a specific input image, our model first produces multi-view images, followed by neural surface construction. To incorporate face geometry information while preserving generalisation to in-the-wild inputs, we estimate a subject-specific mesh directly from the input image, enabling training and evaluation without ground-truth 3D supervision. Importantly, we introduce a multi-view joint generation scheme to enhance the appearance consistency among different views. To the best of our knowledge, this is the first attempt and benchmark for creating photorealistic 3D human face avatars from single images for generic human subject across domains. Extensive experiments demonstrate the efficacy and superiority of our method over previous alternatives for out-of-domain single image 3D face generation and the top ranking competition for the in-domain setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。