仅用一张手绘草图，就能重建出逼真的3D人脸。
Sketch-1-to-3: One Single Sketch to 3D Detailed Face Reconstruction
- 通过新模块增强草图的轮廓与纹理细节提取
- 在真实手绘数据集上实现顶尖重建效果
- 适合图形生成与逆向建模研究者
从单张草图重建3D人脸是一项关键但研究不足的任务，具有广泛实际应用价值。主要挑战在于2D草图与3D人脸结构之间的巨大模态差异：(1) 如何准确从草图中提取面部关键点；(2) 保持多样化的面部表情和精细纹理细节；(3) 在数据有限的情况下训练高性能模型。本文提出Sketch-1-to-3框架，解决上述问题。首先引入几何轮廓与纹理细节（GCTD）模块，提升草图中几何轮廓与纹理特征的提取能力；其次设计包含领域自适应模块和定制损失函数的深度学习架构，实现草图与3D人脸空间的对齐，从而高保真还原表情与纹理。为促进评估与后续研究，构建了真实手绘人脸草图数据集SketchFaces，以及合成草图数据集Syn-SketchFaces。大量实验表明，Sketch-1-to-3在基于草图的3D人脸重建任务中达到当前最优性能。
原文摘要 · Abstract (English)
3D face reconstruction from a single sketch is a critical yet underexplored task with significant practical applications. The primary challenges stem from the substantial modality gap between 2D sketches and 3D facial structures, including: (1) accurately extracting facial keypoints from 2D sketches; (2) preserving diverse facial expressions and fine-grained texture details; and (3) training a high-performing model with limited data. In this paper, we propose Sketch-1-to-3, a novel framework for realistic 3D face reconstruction from a single sketch, to address these challenges. Specifically, we first introduce the Geometric Contour and Texture Detail (GCTD) module, which enhances the extraction of geometric contours and texture details from facial sketches. Additionally, we design a deep learning architecture with a domain adaptation module and a tailored loss function to align sketches with the 3D facial space, enabling high-fidelity expression and texture reconstruction. To facilitate evaluation and further research, we construct SketchFaces, a real hand-drawn facial sketch dataset, and Syn-SketchFaces, a synthetic facial sketch dataset. Extensive experiments demonstrate that Sketch-1-to-3 achieves state-of-the-art performance in sketch-based 3D face reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。