让3D高斯点云直接生成精细轻量级网格,无需后处理。
MILo: Mesh-In-the-Loop Gaussian Splatting for Detailed and Efficient Surface Reconstruction
- 训练时同步优化高斯点与网格,实时生成拓扑结构。
- 重建完整场景,顶点数减少一个数量级,精度达顶尖水平。
- 适合物理模拟、动画等下游应用,网格轻巧无空腔。
尽管近期高斯点云渲染技术已实现从图像快速重建高质量三维场景,但准确提取表面网格仍具挑战。现有方法依赖耗时的后处理步骤,导致细节丢失或生成含数百万顶点的密集网格。更根本的问题在于,从体素到表面表示的后验转换限制了最终网格对训练中捕捉几何结构的保留能力。本文提出MILo,一种新型高斯点云框架,通过可微方式从3D高斯点中直接提取网格,弥合体素与表面表示之间的鸿沟。我们设计了全可微流程,在每次训练迭代中直接由高斯参数生成网格(包括顶点位置与连接关系),这些参数是训练中唯一优化的量。关键贡献包括:双向一致性框架,确保高斯与提取网格在训练中捕获相同几何;每轮迭代进行自适应网格提取,利用高斯作为可微三角剖分枢纽;一种从3D高斯计算有符号距离的新方法,实现精确表面提取且避免几何侵蚀。本方法能以顶尖质量重建完整场景(含背景),所需网格顶点数比此前方法少一个数量级。由于网格轻量且内部为空,非常适用于物理模拟或动画等下游任务。
原文摘要 · Abstract (English)
While recent advances in Gaussian Splatting have enabled fast reconstruction of high-quality 3D scenes from images, extracting accurate surface meshes remains a challenge. Current approaches extract the surface through costly post-processing steps, resulting in the loss of fine geometric details or requiring significant time and leading to very dense meshes with millions of vertices. More fundamentally, the a posteriori conversion from a volumetric to a surface representation limits the ability of the final mesh to preserve all geometric structures captured during training. We present MILo, a novel Gaussian Splatting framework that bridges the gap between volumetric and surface representations by differentiably extracting a mesh from the 3D Gaussians. We design a fully differentiable procedure that constructs the mesh-including both vertex locations and connectivity-at every iteration directly from the parameters of the Gaussians, which are the only quantities optimized during training. Our method introduces three key technical contributions: a bidirectional consistency framework ensuring both representations-Gaussians and the extracted mesh-capture the same underlying geometry during training; an adaptive mesh extraction process performed at each training iteration, which uses Gaussians as differentiable pivots for Delaunay triangulation; a novel method for computing signed distance values from the 3D Gaussians that enables precise surface extraction while avoiding geometric erosion. Our approach can reconstruct complete scenes, including backgrounds, with state-of-the-art quality while requiring an order of magnitude fewer mesh vertices than previous methods. Due to their light weight and empty interior, our meshes are well suited for downstream applications such as physics simulations or animation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。