无需真实纹理标签，用扩散模型恢复高保真人脸贴图。
FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategy
- 分离训练外观与结构网络，推理时交叉组装生成一致贴图
- 在多种姿态和遮挡下仍能还原皱纹、妆容等细节
- 适合需要快速生成高质量人脸贴图的场景
从单张2D图像恢复高质量3D人脸纹理是一项挑战性任务，尤其在数据有限且面部细节复杂（如化妆、皱纹、遮挡）的情况下。本文提出FreeUV，一种无需真实或合成UV数据标注的纹理恢复框架。该方法结合预训练稳定扩散模型与跨组装推理策略，在训练阶段分别优化外观真实性和结构一致性，推理时动态融合二者生成连贯纹理。实验表明，FreeUV在定量与定性指标上均优于现有方法，能准确捕捉复杂面部特征，对多姿态与遮挡具有鲁棒性。此外，该框架支持局部编辑、特征插值及多视角纹理重建等新应用。通过降低数据依赖，为真实场景中生成高保真3D人脸纹理提供了可扩展方案。
原文摘要 · Abstract (English)
Recovering high-quality 3D facial textures from single-view 2D images is a challenging task, especially under constraints of limited data and complex facial details such as makeup, wrinkles, and occlusions. In this paper, we introduce FreeUV, a novel ground-truth-free UV texture recovery framework that eliminates the need for annotated or synthetic UV data. FreeUV leverages pre-trained stable diffusion model alongside a Cross-Assembly inference strategy to fulfill this objective. In FreeUV, separate networks are trained independently to focus on realistic appearance and structural consistency, and these networks are combined during inference to generate coherent textures. Our approach accurately captures intricate facial features and demonstrates robust performance across diverse poses and occlusions. Extensive experiments validate FreeUV's effectiveness, with results surpassing state-of-the-art methods in both quantitative and qualitative metrics. Additionally, FreeUV enables new applications, including local editing, facial feature interpolation, and multi-view texture recovery. By reducing data requirements, FreeUV offers a scalable solution for generating high-fidelity 3D facial textures suitable for real-world scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。