用真实与合成数据训练发型先验,单图重建逼真发丝三维结构。
Im2Haircut: Single-view Strand-based Hair Reconstruction for Human Avatars
- 融合真实与合成数据训练变压器先验,学习发型内外结构。
- 单图输入下重建发丝方向、轮廓和背面一致性均优于现有方法。
- 适合虚拟形象、影视特效中需高精度发型建模的场景。
我们提出一种基于全局发型先验与局部优化的单图像3D发型重建新方法。由于发型多样性和几何复杂性,以及缺乏真实标注训练数据,从单张照片重建发丝级几何极具挑战。传统多视角立体方法仅重建可见发丝,遗漏内部结构,影响真实感模拟。现有方法依赖合成数据训练发型先验,但合成数据量少且质量受限,需专业艺术家手动建模并渲染近似真实图像。为此,我们提出新训练方案:在合成数据上训练变压器先验模型以获取发型内部结构知识,并引入真实数据优化外部结构建模。该方法能准确还原输入图像中的可见发丝,同时保持发型整体三维结构。基于此先验,我们构建了基于高斯溅射的重建方法,可从一张或多张图像生成发型。与现有管道的定性与定量对比显示,本方法在捕捉发丝方向、整体轮廓及背面一致性方面表现更优。
原文摘要 · Abstract (English)
We present a novel approach for 3D hair reconstruction from single photographs based on a global hair prior combined with local optimization. Capturing strand-based hair geometry from single photographs is challenging due to the variety and geometric complexity of hairstyles and the lack of ground truth training data. Classical reconstruction methods like multi-view stereo only reconstruct the visible hair strands, missing the inner structure of hairstyles and hampering realistic hair simulation. To address this, existing methods leverage hairstyle priors trained on synthetic data. Such data, however, is limited in both quantity and quality since it requires manual work from skilled artists to model the 3D hairstyles and create near-photorealistic renderings. To address this, we propose a novel approach that uses both, real and synthetic data to learn an effective hairstyle prior. Specifically, we train a transformer-based prior model on synthetic data to obtain knowledge of the internal hairstyle geometry and introduce real data in the learning process to model the outer structure. This training scheme is able to model the visible hair strands depicted in an input image, while preserving the general 3D structure of hairstyles. We exploit this prior to create a Gaussian-splatting-based reconstruction method that creates hairstyles from one or more images. Qualitative and quantitative comparisons with existing reconstruction pipelines demonstrate the effectiveness and superior performance of our method for capturing detailed hair orientation, overall silhouette, and backside consistency. For additional results and code, please refer to https://im2haircut.is.tue.mpg.de.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。