用预训练3D模型提升单图人脸重建的视角一致性。
KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstruction
- 将预训练3D先验投影到FLAME参数空间，结合2D高斯点云建模外观。
- 在可控与真实场景下均实现更高精度和跨视角一致性。
- 适合需要高保真、多视角一致的人脸重建研究者。
我们提出KaoLRM，旨在将大型重建模型（LRM）的预训练3D先验重新用于单视图图像的参数化3D人脸重建。尽管参数化3D表情模型（3DMM）因紧凑且可解释的参数化被广泛应用，但现有3DMM回归器在不同视角下仍表现不一致。为此，我们利用LRM的预训练3D先验，并将基于FLAME的2D高斯点云融入其渲染流程。具体而言，KaoLRM将LRM的预训练三平面特征投影至FLAME参数空间以恢复几何结构，并通过紧密耦合于FLAME网格的2D高斯原语建模外观。丰富的先验使FLAME回归器能感知3D结构，从而在自遮挡和多样化视角下实现准确稳健的重建。在受控与真实场景基准上的实验表明，KaoLRM在重建精度与跨视角一致性上均优于现有方法，而现有方法对视角变化仍敏感。代码已开源：https://github.com/CyberAgentAILab/KaoLRM。
原文摘要 · Abstract (English)
We propose KaoLRM to re-target the learned prior of the Large Reconstruction Model (LRM) for parametric 3D face reconstruction from single-view images. Parametric 3D Morphable Models (3DMMs) have been widely used for facial reconstruction due to their compact and interpretable parameterization, yet existing 3DMM regressors often exhibit poor consistency across varying viewpoints. To address this, we harness the pre-trained 3D prior of LRM and incorporate FLAME-based 2D Gaussian Splatting into LRM's rendering pipeline. Specifically, KaoLRM projects LRM's pre-trained triplane features into the FLAME parameter space to recover geometry, and models appearance via 2D Gaussian primitives that are tightly coupled to the FLAME mesh. The rich prior enables the FLAME regressor to be aware of the 3D structure, leading to accurate and robust reconstructions under self-occlusions and diverse viewpoints. Experiments on both controlled and in-the-wild benchmarks demonstrate that KaoLRM achieves superior reconstruction accuracy and cross-view consistency, while existing methods remain sensitive to viewpoint variations. The code is released at https://github.com/CyberAgentAILab/KaoLRM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。