实时单目3D人脸重建，速度超快且精度高。
RealDenseFace: Real-time Monocular 3D Face Reconstruction from Dense UV-space Priors
- 用神经网络预测稠密UV图，指导3DMM优化
- 仅数次迭代即可完成高精度重建，速度超20倍提升
- 适合实时追踪与视频序列重建场景
近期单目3D人脸重建方法通过将3D可变形模型（3DMM）拟合到网络预测的稠密先验上实现高保真度，但优化阶段计算成本高，每张图像需数十秒。本文提出RealDenseFace，一种基于优化的实时单目3D人脸重建方法，利用稠密UV空间网络预测。核心思想是将3DMM拟合建模为非线性最小二乘问题，并采用定制化的高斯-牛顿求解器，仅需数次迭代即收敛。重建分两阶段：第一阶段，网络从单张RGB图像中预测两个稠密UV图——用于图像对齐的对应图和沿视角方向的相对深度图；第二阶段，求解器在顶点的UV坐标处采样目标值并进行拟合。该方法支持单图像拟合、离线序列重建与在线跟踪三种场景。在NeRSemble SVFR基准上达到当前最优精度，在线跟踪可达80+ FPS，离线序列重建速度比以往优化基线快20倍以上。
原文摘要 · Abstract (English)
Recent monocular 3D face reconstruction methods achieve high fidelity by fitting a 3D Morphable Model (3DMM) to dense priors predicted by networks, but the optimization stage is computationally expensive, often taking tens of seconds per image. We present RealDenseFace, a real-time optimization-based 3D face reconstruction method with dense UV-space network predictions. Our key idea is to formulate 3DMM fitting as a nonlinear least-squares problem and solve it with a tailored Gauss-Newton solver that converges in only a few iterations. The reconstruction is conducted in two stages. In the first stage, the network predicts two dense UV-space maps from a single RGB image: a correspondence map for UV-to-image alignment, and a relative-depth map for geometric constraints along the viewing direction. In the second stage, the solver fits per-vertex targets sampled from these maps at the vertex UV coordinates. The solver supports all three reconstruction settings: single-image fitting, offline sequence reconstruction, and online tracking. Our method achieves state-of-the-art accuracy on the NeRSemble SVFR benchmark. The online tracker runs at 80+ FPS, and the offline sequence reconstruction is over 20 times faster than previous optimization-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。