提出可分离发型与面部的3D头像通用先验模型
HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars
- 将人脸与发型分开学习潜在空间,引入组合性先验
- 在有限数据下实现发型与面部的自然解耦
- 支持换脸换发型,适合个性化3D avatar生成
我们提出一种显式考虑发型与面部组合性的3D头像通用先验模型。现有方法多采用整体建模,将面部与发型视为不可分割的整体,忽视了头部的内在组合性,导致在数据有限时难以自然解耦面部与发型表示,且不支持灵活可控的3D人脸与发型互换。为此,我们设计了一种合成无发数据的生成流水线,利用扩散先验估计无发几何与纹理,从工作室采集的数据中移除头发。基于配对的有发与无发数据,我们训练了分离的面部与发型先验模型,并将组合性作为归纳偏置以促进有效解耦。该模型天然具备组合性,可无缝迁移面部与发型组件,同时保持身份一致。此外,我们展示了仅需少量单目图像即可微调生成高保真、可组合发型的3D头像,适用于未见个体。该方法在真实场景中具有良好的实用性,为灵活、生动的3D头像生成开辟新路径。
原文摘要 · Abstract (English)
We present a universal prior model for 3D head avatars with explicit hair compositionality. Existing approaches to build generalizable priors for 3D head avatars often adopt a holistic modeling approach, treating the face and hair as an inseparable entity. This overlooks the inherent compositionality of the human head, making it difficult for the model to naturally disentangle face and hair representations, especially when the dataset is limited. Furthermore, such holistic models struggle to support applications like 3D face and hairstyle swapping in a flexible and controllable manner. To address these challenges, we introduce a prior model that explicitly accounts for the compositionality of face and hair, learning their latent spaces separately. A key enabler of this approach is our synthetic hairless data creation pipeline, which removes hair from studio-captured datasets using estimated hairless geometry and texture derived from a diffusion prior. By leveraging a paired dataset of hair and hairless captures, we train disentangled prior models for face and hair, incorporating compositionality as an inductive bias to facilitate effective separation. Our model's inherent compositionality enables seamless transfer of face and hair components between avatars while preserving identity. Additionally, we demonstrate that our model can be fine-tuned in a few-shot manner using monocular captures to create high-fidelity, hair-compositional 3D head avatars for unseen subjects. These capabilities highlight the practical applicability of our approach in real-world scenarios, paving the way for flexible and expressive 3D avatar generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。