通过多线索融合提升3D高斯溅射的实例一致性
Consistent Scene Understanding in 3D Gaussian Splatting via Multi-Cue Mask Refinement
- 融合语义、几何与结构线索生成协同先验
- 跨视角匹配实现全局一致的实例标识
- 显著提升分割稳定性与下游编辑效果
可靠的事例级场景理解是实现物体级交互和高保真3D表示的基础。现有方法常依赖2D基础分割模型获取先验，但其2D中心设计通常导致掩码碎片化及多视角预测不一致。为此，我们提出一种新框架，通过生成一致的2D实例掩码来指导3D高斯溅射（3DGS）特征场优化。框架包含三个阶段：(1) 多线索提取，从输入图像中生成语义、几何与结构先验；(2) 多线索引导的掩码合并，利用语义、深度和边缘线索构成的综合得分整合碎片化掩码；(3) 跨视角掩码匹配，建立所有视角间的全局一致身份分配。该方法将视角相关分割转化为连贯3D原语，实现稳定3D实例分割并支持有效下游编辑。实验表明，本方法在跨视角一致性与分割稳定性上显著优于现有基线，同时保持高保真光度重建。
原文摘要 · Abstract (English)
Reliable instance-level scene understanding is a fundamental prerequisite for object-level interactions and high-fidelity 3D representations. While current methods often leverage 2D foundation segmentation models to obtain these priors, their 2D-centric design typically yields fragmented masks and inconsistent predictions across different views. To address these issues, we propose a novel framework that produces consistent 2D instance masks to guide the optimization of 3D Gaussian Splatting (3DGS) feature fields. Our framework consists of three main stages. (1) Multi-Cue Extraction that generates synergistic semantic, geometric, and structural priors from input images. (2) Multi-Cue-Guided Mask Merging process that consolidates fragmented masks using a composite merge score derived from semantic, depth, and edge cues. (3) Cross-View Mask Matching that establishes globally consistent identity assignments across all viewpoints. By transforming viewpoint-specific segments into coherent 3D primitives, our approach enables stable 3D instance segmentation and effective downstream editing tasks. Experiments demonstrate that our method significantly improves cross-view consistency and segmentation stability over existing baselines while maintaining high-fidelity photometric reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。