GLS统一优化3D重建与开放词汇分割，提升细节与平滑性。
GLS: Geometry-aware 3D Language Gaussian Splatting
- 引入法向量先验引导渲染法向，提升深度优化精度。
- 结合CLIP特征与DEVA掩码，在多视角下保持语义一致性。
- 在三个数据集上超越现有方法，兼顾重建与分割性能。
近期，3D高斯点阵（3DGS）在室内表面重建和3D开放词汇分割任务中表现优异。本文提出GLS，一个基于3DGS的统一框架，同时实现3D表面重建与开放词汇分割。为提升室内重建质量，引入表面法向先验作为几何线索，指导渲染法向，并利用法向误差优化渲染深度；在3D开放词汇分割方面，采用2D CLIP特征引导实例特征，增强表面平滑性，并使用DEVA掩码保持视角一致性。大量实验表明，联合优化表面重建与3D开放词汇分割有效，GLS在MuSHRoom、ScanNet++和LERF-OVS数据集上均优于各任务当前最优方法。
原文摘要 · Abstract (English)
Recently, 3D Gaussian Splatting (3DGS) has achieved impressive performance on indoor surface reconstruction and 3D open-vocabulary segmentation. This paper presents GLS, a unified framework of 3D surface reconstruction and open-vocabulary segmentation based on 3DGS. GLS extends two fields by improving their sharpness and smoothness. For indoor surface reconstruction, we introduce surface normal prior as a geometric cue to guide the rendered normal, and use the normal error to optimize the rendered depth. For 3D open-vocabulary segmentation, we employ 2D CLIP features to guide instance features and enhance the surface smoothness, then utilize DEVA masks to maintain their view consistency. Extensive experiments demonstrate the effectiveness of jointly optimizing surface reconstruction and 3D open-vocabulary segmentation, where GLS surpasses state-of-the-art approaches of each task on MuSHRoom, ScanNet++ and LERF-OVS datasets. Project webpage: https://jiaxiongq.github.io/GLS_ProjectPage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。