从单张照片重建无眼镜的3D人脸,解决野外场景难题
Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction
- 通过检测眼镜区域并智能移除,生成更真实的3D面部结构
- 基于2D图像直接回归3DMM参数,保持眼鼻嘴拓扑关系真实
- 可融合面部解析任务提升重建质量,适合真实场景应用
单视图3D人脸重建是计算机视觉中极具挑战性的问题。现有方法通常假设输入为无遮挡人脸,难以适应真实环境。本文提出一种从单张图像中去除眼镜并重建逼真3D人脸的方法。核心创新在于鲁棒地识别眼镜区域,并智能移除以构建3D纹理。通过估计合理位置的眼镜区域2D结构,辅助3D纹理生成。所提方法采用深度学习架构,直接从单张2D图像回归3DMM表示,确保眼、鼻、口间拓扑结构真实可信。同时证明了将相关面部解析任务融入框架可进一步提升重建质量。在多个现有3D人脸重建任务上进行大量实验,验证了本方法对复杂场景的强适应能力,优于现有方法。
原文摘要 · Abstract (English)
Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the input is unobstructed faces which makes their method not suitable for in-the-wild conditions. We present a method for performing a 3D face that removes eyeglasses from a single image. Existing facial reconstruction methods fail to remove eyeglasses automatically for generating a photo-realistic 3D face "in-the-wild".The innovation of our method lies in a process for identifying the eyeglasses area robustly and remove it intelligently. In this work, we estimate the 2D face structure of the reasonable position of the eyeglasses area, which is used for the construction of 3D texture. An excellent anti-eyeglasses face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We achieve this via a deep learning architecture that performs direct regression of a 3DMM representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related face parsing task can be incorporated into the proposed framework and help improve reconstruction quality. We conduct extensive experiments on existing 3D face reconstruction tasks as concrete examples to demonstrate the method's superior regulation ability over existing methods often break down.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。