arXiv:2503.21886cs.GRcs.CV2025-03

用精修网格提升单目视频人脸重建质量,效果优于当前最佳方法。

Refined Geometry-guided Head Avatar Reconstruction from Monocular RGB Video

  • 分两阶段重建:先用3DMM模板生成初始网格,再通过SDF优化细节。
  • 引入拉普拉斯平滑降低NeRF密度场噪声,保留3DMM特征的同时细化几何。
  • 适合追求高保真虚拟人形象的开发者和研究人员使用。

从单目视频高保真重建头部化身在虚拟人应用中极具价值,但仍是计算机图形学与计算机视觉领域的挑战。本文提出一种两阶段头部化身重建网络,采用精细化3D网格表示。不同于依赖3DMM粗略模板的方法,本方法旨在学习适配NeRF的精细网格以捕捉复杂面部细节。第一阶段,基于3DMM存储的NeRF与初始网格训练,利用几何先验并用一致的隐码融合多帧观测。第二阶段,基于初始NeRF密度场构建的SDF,提出新颖的网格精化流程。为消除NeRF密度场常见噪声而不损失3DMM特征,对位移场施加拉普拉斯平滑。随后使用精化网格进行第二阶段训练,引导网络学习更细微的面部特征。实验表明,该方法显著提升基于初始网格的NeRF渲染效果,在高保真头部化身重建上超越现有最先进方法。

原文摘要 · Abstract (English)

High-fidelity reconstruction of head avatars from monocular videos is highly desirable for virtual human applications, but it remains a challenge in the fields of computer graphics and computer vision. In this paper, we propose a two-phase head avatar reconstruction network that incorporates a refined 3D mesh representation. Our approach, in contrast to existing methods that rely on coarse template-based 3D representations derived from 3DMM, aims to learn a refined mesh representation suitable for a NeRF that captures complex facial nuances. In the first phase, we train 3DMM-stored NeRF with an initial mesh to utilize geometric priors and integrate observations across frames using a consistent set of latent codes. In the second phase, we leverage a novel mesh refinement procedure based on an SDF constructed from the density field of the initial NeRF. To mitigate the typical noise in the NeRF density field without compromising the features of the 3DMM, we employ Laplace smoothing on the displacement field. Subsequently, we apply a second-phase training with these refined meshes, directing the learning process of the network towards capturing intricate facial details. Our experiments demonstrate that our method further enhances the NeRF rendering based on the initial mesh and achieves performance superior to state-of-the-art methods in reconstructing high-fidelity head avatars with such input.

人脸重建NeRF网格优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。