PS-GS联合优化几何、材质与光照,实现多视角多光源高精度3D重建。
PS-GS: Gaussian Splatting for Multi-View Photometric Stereo
- 基于2D高斯点云初始化,通过延迟反向渲染联合估计几何、材质和光照。
- 在合成与真实数据集上,重建精度优于现有方法,计算效率更高。
- 适合需要高保真3D重建与自由编辑的场景,如虚拟试穿、数字孪生。
将逆渲染与多视角光度立体(MVPS)结合,可获得比依赖固定环境光照的逆渲染方法更准确的3D重建结果。然而,高效实现基于MVPS的逆渲染仍具挑战。为此,我们提出多光源光度立体高斯点云(PS-GS),能够高效联合估计被多个方向光源照射物体的几何、材质与光照。首先,重建标准2D高斯点云作为初始几何;随后,基于该模型,通过包含光照计算的多层感知机执行延迟逆渲染。在整个优化过程中,利用未校准光度立体估计的法向图对渲染法向图进行正则化,并提出针对单向光源的2D高斯光线追踪以精炼入射光照。多视图与多光源图像的使用缓解了逆渲染的病态问题。优化完成后,重建对象可用于新视角合成、再光照及材质与形状编辑。在合成与真实数据集上的实验表明,本方法在重建精度与计算效率方面均优于现有工作。
原文摘要 · Abstract (English)
Integrating inverse rendering with multi-view photometric stereo (MVPS) yields more accurate 3D reconstructions than the inverse rendering approaches that rely on fixed environment illumination. However, efficient inverse rendering with MVPS remains challenging. To fill this gap, we introduce the Gaussian Splatting for Multi-view Photometric Stereo (PS-GS), which efficiently and jointly estimates the geometry, materials, and lighting of the object that is illuminated by diverse directional lights (multi-light). Our method first reconstructs a standard 2D Gaussian splatting model as the initial geometry. Based on the initialization model, it then proceeds with the deferred inverse rendering by the full rendering equation containing a lighting-computing multi-layer perceptron. During the whole optimization, we regularize the rendered normal maps by the uncalibrated photometric stereo estimated normals. We also propose the 2D Gaussian ray-tracing for single directional light to refine the incident lighting. The regularizations and the use of multi-view and multi-light images mitigate the ill-posed problem of inverse rendering. After optimization, the reconstructed object can be used for novel-view synthesis, relighting, and material and shape editing. Experiments on both synthetic and real datasets demonstrate that our method outperforms prior works in terms of reconstruction accuracy and computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。