arXiv:2508.08219cs.CV2025-08被引 2

实时分割3D高斯点云,无需训练即可跨视角保持一致。

SAGOnline: Segment Any Gaussians Online

  • 将分割任务拆解为轻量子任务,结合视频基础模型生成时序一致的2D掩码。
  • 在NVOS和SPIn-NeRF上分别达到92.7%和95.2%的mIoU,每帧仅需27ms。
  • 支持即时提示、实例与语义分割,适合AR/VR与机器人交互场景。

3D高斯点阵已成为显式三维场景表示的强大范式,但实现高效且一致的3D分割仍具挑战。现有方法通常依赖高维特征提升,导致优化成本高、语义隐式且任务特定。本文提出统一的零样本框架SAGOnline,实现无需场景训练的实时跨视角一致分割。SAGOnline将整体分割问题解耦为轻量级子任务,通过集成视频基础模型(如SAM 2)生成渲染视图间的时序一致2D掩码。关键在于,不学习连续特征场,而是引入栅格化感知几何共识机制,利用高斯栅格化流水线的可追溯性,确定性地将2D预测映射到显式的离散3D原始标签,实现毫秒级推理。该离散表示免除了特征蒸馏的内存与计算开销。在NVOS和SPIn-NeRF基准上的评估表明,SAGOnline在准确率(92.7%和95.2% mIoU)与速度(27ms/帧)上均达当前最优。其灵活接口支持即时提示、实例与语义分割,为AR/VR与机器人中的交互式3D理解铺平道路。

原文摘要 · Abstract (English)

3D Gaussian Splatting has emerged as a powerful paradigm for explicit 3D scene representation, yet achieving efficient and consistent 3D segmentation remains challenging. Existing segmentation approaches typically rely on high-dimensional feature lifting, which causes costly optimization, implicit semantics, and task-specific constraints. We present \textbf{Segment Any Gaussians Online (SAGOnline)}, a unified, zero-shot framework that achieves real-time, cross-view consistent segmentation without scene-specific training. SAGOnline decouples the monolithic segmentation problem into lightweight sub-tasks. By integrating video foundation models (e.g., SAM 2), we first generate temporally consistent 2D masks across rendered views. Crucially, instead of learning continuous feature fields, we introduce a \textbf{Rasterization-aware Geometric Consensus} mechanism that leverages the traceability of the Gaussian rasterization pipeline. This allows us to deterministically map 2D predictions to explicit, discrete 3D primitive labels in real-time. This discrete representation eliminates the memory and computational burden of feature distillation, enabling instant inference. Extensive evaluations on NVOS and SPIn-NeRF benchmarks demonstrate that SAGOnline achieves state-of-the-art accuracy (92.7\% and 95.2\% mIoU) while operating at the fastest speed at 27 ms per frame. By providing a flexible interface for diverse foundation models, our framework supports instant prompt, instance, and semantic segmentation, paving the way for interactive 3D understanding in AR/VR and robotics.

3D分割高斯点阵实时推理AR/VR

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。