用3D高斯点云重建实现无监督点云分割,提升语义一致性。
PointGS: Semantic-Consistent Unsupervised 3D Point Cloud Segmentation with 3D Gaussian Splatting

- 将点云转为3D高斯空间,填补空缺并建模遮挡关系。
- 通过多视角渲染与SAM结合,使2D语义迁移至3D保持一致。
- 适用于缺乏标注数据的自动驾驶与机器人场景。
无监督点云分割对具身人工智能和自动驾驶至关重要,可避免全监督方法所需的密集点级标注成本。尽管利用2D预训练模型(如SAM)补充语义信息是自然选择,但离散3D点与连续2D图像之间存在根本性不匹配,导致投影重叠和复杂模态对齐问题,从而削弱2D-3D迁移中的语义一致性。为此,本文提出PointGS,一种简单有效的无监督3D点云分割流程。PointGS采用3D高斯点阵作为统一中间表示,通过多视角观测将稀疏点云重建为稠密3D高斯空间,填补空间间隙并编码遮挡关系,消除投影引起的语义混淆。从高斯空间渲染多视角稠密图像,使用SAM提取2D语义掩码,并通过对比学习将语义蒸馏至3D高斯原语,确保不同视角间语义分配一致。通过两步配准将高斯空间与原始点云对齐,再以最近邻搜索为点云分配语义。实验表明,PointGS优于现有最先进无监督方法,在ScanNet-V2上提升0.9% mIoU,S3DIS上提升2.8% mIoU。
原文摘要 · Abstract (English)
Unsupervised point cloud segmentation is critical for embodied artificial intelligence and autonomous driving, as it mitigates the prohibitive cost of dense point-level annotations required by fully supervised methods. While integrating 2D pre-trained models such as the Segment Anything Model (SAM) to supplement semantic information is a natural choice, this approach faces a fundamental mismatch between discrete 3D points and continuous 2D images. This mismatch leads to inevitable projection overlap and complex modality alignment, resulting in compromised semantic consistency across 2D-3D transfer. To address these limitations, this paper proposes PointGS, a simple yet effective pipeline for unsupervised 3D point cloud segmentation. PointGS leverages 3D Gaussian Splatting as a unified intermediate representation to bridge the discrete-continuous domain gap. Input sparse point clouds are first reconstructed into dense 3D Gaussian spaces via multi-view observations, filling spatial gaps and encoding occlusion relationships to eliminate projection-induced semantic conflation. Multi-view dense images are rendered from the Gaussian space, with 2D semantic masks extracted via SAM, and semantics are distilled to 3D Gaussian primitives through contrastive learning to ensure consistent semantic assignments across different views. The Gaussian space is aligned with the original point cloud via two-step registration, and point semantics are assigned through nearest-neighbor search on labeled Gaussians. Experiments demonstrate that PointGS outperforms state-of-the-art unsupervised methods, achieving +0.9% mIoU on ScanNet-V2 and +2.8% mIoU on S3DIS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。