用高斯点云实现2D标签向3D的秒级迁移,无需专门训练。
CDSeg: A Renderable Gaussian Carrier for Image-to-3D Label Transfer

- 以高斯点为可渲染载体,通过多视角投票融合2D标签
- 在桌面、扫描场景上达92%以上分割准确率,百万级点云秒级处理
- 支持提示、实例、语义等多重标签,适合快速部署3D标注
现代图像模型能提供每个视角中应分割的内容线索,但其掩码无法决定这些标签在3D空间中的位置。我们提出跨域分割方法CDSeg,无需特定任务的3D分割训练,使用高斯点作为可渲染的标签载体。外部掩码源提供标签,渲染生成的可见性决定哪些3D高斯点接收标签。载体可通过将输入点补全为高斯点(保留索引)或复用优化后高斯场景的原始点来构建。CDSeg在渲染过程中记录像素-点关联,通过投票与局部滤波融合多视角掩码。生成的标签可返回原点、保留在原始高斯场景中,或渲染至其他视角。CDSeg支持可提示、自动实例、语义及LiDAR设置,可在数秒内处理含百万级点的场景。在DesktopObjects-360上达到92.35% mIoU,NeRDS-360上为95.89%,完整ScanNet-v2验证集上为65.77%,均基于提供的2D语义标注。该方法提供统一接口,实现2D掩码在点云、高斯场景与图像视图间的高效重用,无需专用3D分割网络。
原文摘要 · Abstract (English)
Modern image models provide strong cues about \emph{what} should be segmented in each view, but their masks do not by themselves determine \emph{where} those labels should persist in 3D. We present Cross-Domain Segmentation via Gaussian Splatting (CDSeg), a label-transfer interface that requires no task-specific 3D segmentation training and uses Gaussian primitives as a renderable label carrier. An external mask source supplies the labels, while renderer-derived visibility determines which 3D primitives receive them. The carrier is instantiated either by completing each input point into one Gaussian, preserving its index, or by reusing the native primitives of an optimized Gaussian scene. CDSeg records pixel--primitive associations during rendering and fuses multi-view masks through voting and a local filter. The resulting labels can be returned to the original points, retained on the native Gaussian scene, or rendered into other views. CDSeg covers promptable, automatic instance, semantic, and LiDAR settings and processes scenes with millions of primitives in seconds. It obtains 92.35\% mIoU on DesktopObjects-360, 95.89\% on NeRDS-360, and 65.77\% on the full ScanNet-v2 validation split using the provided 2D semantic annotations. CDSeg thereby provides one interface for reusing 2D masks across point clouds, Gaussian scenes, and image views without a task-specific 3D segmentation network.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。