用可微分拉拽实现3D高斯与SDF的协同优化,提升表面重建精度。
Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set
- 将3D高斯动态拉到SDF零等值面,实现几何一致性约束
- 联合优化高斯和SDF,使表面更完整、光滑且细节丰富
- 适合需要高精度三维重建的研究者或工业应用
基于多视角的表面重建中,推断符号距离函数(SDF)至关重要。3D高斯点阵(3DGS)在体渲染方面提供了新视角,具有渲染效率和质量优势。然而,由于3D高斯的离散性、稀疏性及偏离表面的问题,难以利用3DGS推断SDF。为此,我们提出一种方法,无缝融合3DGS与神经SDF学习。核心思想是通过多视角一致性更有效地约束SDF推断。具体地,使用神经拉拽将3D高斯动态对齐至神经SDF的零等值面,并通过可微分光栅化渲染对齐后的高斯。同时,通过将邻近空间拉向被拉拽的高斯来更新神经SDF,逐步精炼表面附近的符号距离场。结合可微分拉拽与点阵渲染,我们联合优化3D高斯与神经SDF,同时满足RGB与几何约束,恢复出更精确、平滑且完整的表面,包含更多几何细节。数值与视觉对比表明,该方法在广泛使用的基准上优于当前最先进结果。
原文摘要 · Abstract (English)
It is vital to infer a signed distance function (SDF) in multi-view based surface reconstruction. 3D Gaussian splatting (3DGS) provides a novel perspective for volume rendering, and shows advantages in rendering efficiency and quality. Although 3DGS provides a promising neural rendering option, it is still hard to infer SDFs for surface reconstruction with 3DGS due to the discreteness, the sparseness, and the off-surface drift of 3D Gaussians. To resolve these issues, we propose a method that seamlessly merge 3DGS with the learning of neural SDFs. Our key idea is to more effectively constrain the SDF inference with the multi-view consistency. To this end, we dynamically align 3D Gaussians on the zero-level set of the neural SDF using neural pulling, and then render the aligned 3D Gaussians through the differentiable rasterization. Meanwhile, we update the neural SDF by pulling neighboring space to the pulled 3D Gaussians, which progressively refine the signed distance field near the surface. With both differentiable pulling and splatting, we jointly optimize 3D Gaussians and the neural SDF with both RGB and geometry constraints, which recovers more accurate, smooth, and complete surfaces with more geometry details. Our numerical and visual comparisons show our superiority over the state-of-the-art results on the widely used benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。