arXiv:2409.08270cs.CVcs.AI2024-09ECCV被引 72

用线性规划实现3D高斯点云分割,30秒完成且精度更高。

FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally

论文配图:FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally
图 1 · 摘自论文原文
  • 基于渲染的线性特性,用线性规划一步求解最优标签分配
  • 30秒内完成分割,比现有方法快50倍以上
  • 对噪声鲁棒,适合物体移除与修复等下游任务

本文解决从2D掩码准确分割3D高斯点云(3D-GS)的难题。传统方法依赖迭代梯度下降为每个高斯分配唯一标签,导致优化耗时长且结果次优。本文提出一种简单却全局最优的求解器:在重建的3D-GS场景下,2D掩码的渲染本质上是各高斯标签的线性函数,因此可通过闭式线性规划求解最优标签分配。该方法利用点云绘制中的透明混合特性实现单步优化,并在目标函数中引入背景偏置,显著提升对噪声的鲁棒性。实验表明,该方法可在30秒内完成分割,较最佳现有方法提速约50倍,适用于多种场景,且在物体移除、图像修补等下游任务中表现更优。演示与代码将开源于https://github.com/florinshen/FlashSplat。

原文摘要 · Abstract (English)

This study addresses the challenge of accurately segmenting 3D Gaussian Splatting from 2D masks. Conventional methods often rely on iterative gradient descent to assign each Gaussian a unique label, leading to lengthy optimization and sub-optimal solutions. Instead, we propose a straightforward yet globally optimal solver for 3D-GS segmentation. The core insight of our method is that, with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially a linear function with respect to the labels of each Gaussian. As such, the optimal label assignment can be solved via linear programming in closed form. This solution capitalizes on the alpha blending characteristic of the splatting process for single step optimization. By incorporating the background bias in our objective function, our method shows superior robustness in 3D segmentation against noises. Remarkably, our optimization completes within 30 seconds, about 50$\times$ faster than the best existing methods. Extensive experiments demonstrate the efficiency and robustness of our method in segmenting various scenes, and its superior performance in downstream tasks such as object removal and inpainting. Demos and code will be available at https://github.com/florinshen/FlashSplat.

3D分割高斯点云线性规划实时优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。