破解复杂光照下人脸纹理建模难题，实现遮挡场景的精准还原。
Learning to Decouple the Lights for 3D Face Texture Modeling
- 将复杂光照分解为多光源组合，用神经网络学习分离光照影响。
- 在带遮挡的单图与视频上均显著提升纹理重建精度。
- 适合研究真实场景下3D人脸建模、影视特效与虚拟人开发的团队。
现有研究在光照良好且无外部遮挡的人脸图像上已取得显著进展，但在复杂光照与外部遮挡（如戴帽子）场景下，仍难以准确恢复面部纹理。现有方法基于单一均匀光照假设，无法处理此类数据。本文提出一种新方法，通过学习将非自然光照解耦为多个独立光条件的组合，并结合神经表示进行建模，称为「光照解耦」（Light Decoupling）。在单图与视频序列上的实验表明，该方法在受遮挡和复杂光照条件下能有效提升3D人脸纹理重建质量。更多视频与代码请见：https://tianxinhuang.github.io/projects/Deface。
原文摘要 · Abstract (English)
Existing research has made impressive strides in reconstructing human facial shapes and textures from images with well-illuminated faces and minimal external occlusions. Nevertheless, it remains challenging to recover accurate facial textures from scenarios with complicated illumination affected by external occlusions, e.g. a face that is partially obscured by items such as a hat. Existing works based on the assumption of single and uniform illumination cannot correctly process these data. In this work, we introduce a novel approach to model 3D facial textures under such unnatural illumination. Instead of assuming single illumination, our framework learns to imitate the unnatural illumination as a composition of multiple separate light conditions combined with learned neural representations, named Light Decoupling. According to experiments on both single images and video sequences, we demonstrate the effectiveness of our approach in modeling facial textures under challenging illumination affected by occlusions. Please check https://tianxinhuang.github.io/projects/Deface for our videos and codes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。