给3D高斯点云添加语义标签,实现高效精准的场景分割。
LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation
- 在高斯点云中引入语义标签,通过跨视角一致性约束提升分割精度。
- 训练速度比SOTA快22倍,在1440×1080分辨率下实现高效优化。
- 适合需要精确3D场景理解的自动驾驶、机器人导航等应用。
3D高斯溅射(3DGS)作为一种新型显式三维场景表示方法,兼具高保真重建与高效渲染能力。然而,3DGS缺乏3D分割能力,限制了其在需场景理解任务中的应用。准确识别和分离特定物体组件至关重要。为此,我们提出标签感知的3D高斯溅射(LabelGS),在高斯表示中引入对象标签。LabelGS为3D高斯引入跨视角一致的语义掩码,并设计新颖的遮挡分析模型以避免优化过程中的过拟合问题;通过主高斯标签模型将2D语义先验迁移到3D高斯,利用高斯投影滤波器避免标签冲突。该方法实现了高斯表示的有效解耦,并通过随机区域采样策略优化3DGS流程,显著提升效率。大量实验表明,LabelGS在3D场景分割任务中优于此前最优方法,包括Feature-3DGS。值得注意的是,在1440×1080分辨率下,LabelGS训练速度较Feature-3DGS提升22倍。代码将发布于https://github.com/garrisonz/LabelGS。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has emerged as a novel explicit representation for 3D scenes, offering both high-fidelity reconstruction and efficient rendering. However, 3DGS lacks 3D segmentation ability, which limits its applicability in tasks that require scene understanding. The identification and isolating of specific object components is crucial. To address this limitation, we propose Label-aware 3D Gaussian Splatting (LabelGS), a method that augments the Gaussian representation with object label.LabelGS introduces cross-view consistent semantic masks for 3D Gaussians and employs a novel Occlusion Analysis Model to avoid overfitting occlusion during optimization, Main Gaussian Labeling model to lift 2D semantic prior to 3D Gaussian and Gaussian Projection Filter to avoid Gaussian label conflict. Our approach achieves effective decoupling of Gaussian representations and refines the 3DGS optimization process through a random region sampling strategy, significantly improving efficiency. Extensive experiments demonstrate that LabelGS outperforms previous state-of-the-art methods, including Feature-3DGS, in the 3D scene segmentation task. Notably, LabelGS achieves a remarkable 22X speedup in training compared to Feature-3DGS, at a resolution of 1440X1080. Our code will be at https://github.com/garrisonz/LabelGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。