arXiv:2607.18801cs.CV2026-07中稿 · ECCV被引 1

让3D高斯点云能理解任意数量的语义指令,且无需训练和额外存储。

ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting

论文配图:ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting
图 1 · 摘自论文原文
  • 通过多视角几何约束将2D视觉语言模型先验迁移到3D空间
  • 在新构建的GR-LERF和GR-ScanNet数据集上超越现有方法
  • 零训练、零特征存储,适合实时交互式3D场景理解

3D高斯点云(3DGS)的进展使语言引导的场景理解成为可能,但现有针对3D高斯点云的指代分割(R3DGS)方法仅支持单目标查询。为反映真实指令的模糊性,我们提出广义指代3D高斯点云分割(GR3DGS)任务,要求动态分割任意数量的目标(0、1或$N$)。为此,我们构建了两个新基准:GR-LERF与GR-ScanNet。现有R3DGS范式存在根本性技术瓶颈:仅基于2D渲染像素操作,缺乏内在的3D点级理解;且需对每个场景进行优化以嵌入重型语义特征,带来高昂计算开销。为此,我们提出ZeroSplat,一种无训练、零特征的新型框架。通过强健的多视角几何约束,将2D视觉语言模型先验迁移至3D空间,实现原生点级理解且不增加任何特征存储。大量实验表明,ZeroSplat在广义与单目标场景中均显著优于现有方法,同时保持极高效能。

原文摘要 · Abstract (English)

Recent advancements in 3D Gaussian Splatting (3DGS) have enabled language-guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single-target queries. To reflect the ambiguity of real-world instructions, we introduce the Generalized Referring 3D Gaussian Splatting Segmentation (GR3DGS) task, which requires dynamically segmenting an arbitrary number of targets (0, 1, or $N$). To facilitate comprehensive evaluation of this new task, we construct two new benchmarks: GR-LERF and GR-ScanNet. Crucially, existing R3DGS paradigms exhibit fundamental technical bottlenecks that severely limit their performance on the GR3DGS task: they lack intrinsic 3D point-level understanding by operating merely on 2D rendered pixels, and they incur prohibitive computational overhead by requiring per-scene optimization to embed heavy semantic features. To dismantle these bottlenecks, we propose ZeroSplat, a novel training-free and zero-feature framework. ZeroSplat lifts 2D Vision-Language Model (VLM) priors into 3D space through robust multi-view geometric constraints. This strategy enables intrinsic point-level understanding without incurring any additional feature storage. Extensive experiments demonstrate that ZeroSplat significantly outperforms state-of-the-art methods across generalized and single-target scenarios while maintaining exceptional efficiency. Project Page: https://inkmind-ai.github.io/ZeroSplat

3D分割多模态高斯点云零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。