让3D高斯点具备物体身份,提升场景理解能力。
OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives

- 用双透明度机制区分视觉存在与物体占据,避免标签污染。
- 在多个数据集上实现媲美特征法的开放词汇性能,计算开销更低。
- 适合需要精准物体分割与高效推理的3D重建任务。
3D高斯泼溅(3DGS)提供显式高效的场景表示,但其基本单元缺乏固有的物体级身份,阻碍了开放词汇场景理解等下游任务。现有方法通常通过将高维特征嵌入高斯或借助启发式优化将2D掩码标签提升至3D来解决,但特征法带来巨大存储与解码开销,而提升法易受标签污染:用于外观重建的高斯常在2D到3D投影中被错误标注。本文提出OP2GS,一种物体感知的高斯表示,为每个基本单元增加显式的实例身份和专用实例透明度σ*以支持物体掩码渲染。原始透明度σ仍负责视觉重建,而σ*建模高斯是否应贡献于特定物体掩码。该双透明度设计解耦了视觉存在与实例占用:误标高斯可保留在图像渲染中,但在物体掩码分支中变为透明。为学习此表示,引入随机物体损失,利用3DGS的标准透射率可见性优化1维实例占用场。语义描述通过多视角聚合在物体层面附加,消除每高斯特征存储。相比特征训练方法,OP2GS在开放词汇任务上表现相当且显著降低计算开销;相比无训练管道,其利用物理一致的占用学习解决了可见性模糊问题。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) provides an explicit and efficient scene representation, but its primitives lack inherent object-level identity, hindering downstream tasks such as open-vocabulary scene understanding. Existing methods typically address this by either distilling high-dimensional feature embeddings into Gaussians or by lifting 2D mask labels into 3D via heuristic refinement. However, feature-based approaches incur heavy storage and decoding overhead, while lifting-based pipelines remain vulnerable to label contamination: Gaussians necessary for appearance reconstruction often receive incorrect object labels during 2D-to-3D projection. We propose OP2GS, an object-aware Gaussian representation that augments each primitive with an explicit instance identity and a dedicated instance opacity $σ^{*}$ for object-mask rendering. The original opacity $σ$ remains responsible for visual reconstruction, while $σ^{*}$ models whether a Gaussian should contribute to a particular object mask. This dual-opacity formulation decouples visual existence from instance occupancy: mislabeled Gaussians can remain available for image rendering while becoming transparent in the object-mask branch. To learn this representation, we introduce a random object loss that optimizes the 1D instance occupancy field using the standard transmittance-based visibility of 3DGS. Semantic descriptors are then attached at the object level through multi-view aggregation, eliminating per-Gaussian feature storage. Compared with feature-training approaches, OP2GS achieves competitive open-vocabulary performance while significantly reducing computational overhead. Compared with training-free pipelines, it leverages physically consistent occupancy learning to resolve visibility ambiguities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。