arXiv:2602.21552cs.CV2026-02中稿 · CVPR被引 7

用视觉几何先验提升单目3D占据预测精度与速度

Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction

  • 将表面点沿相机射线内扩为高斯体素,实现概率占据推理
  • 单目设置下mIoU提升9.99,流式输入下提升11.79
  • 无需训练即可增量更新,比基线快2.65倍

准确的3D场景理解对具身智能至关重要,占据预测已成为推理物体与自由空间的关键任务。现有方法主要依赖深度先验(如DepthAnything),但对3D线索利用有限,限制了性能与泛化能力。近期的视觉几何模型(如VGGT)展现出强大的3D先验能力,但与单目深度基础模型类似,仍仅作用于可见表面而非体积分内部。为此,我们提出GPOcc框架,利用可泛化的视觉几何先验(GPs)进行单目占据预测。方法将表面点沿相机射线向内扩展生成体积分样本,并以高斯原型表示,实现概率占据推断。为支持流式输入,设计无需训练的增量更新策略,将每帧高斯融合为统一全局表示。在Occ-ScanNet和EmbodiedOcc-ScanNet上的实验表明:相比先前最优方法,单目设置下mIoU提升+9.99,流式设置下提升+11.79;在相同深度先验下,mIoU提升+6.73且运行速度提升2.65倍。结果表明GPOcc更高效地利用了几何先验。代码将发布于https://github.com/JuIvyy/GPOcc。

原文摘要 · Abstract (English)

Accurate 3D scene understanding is essential for embodied intelligence, with occupancy prediction emerging as a key task for reasoning about both objects and free space. Existing approaches largely rely on depth priors (e.g., DepthAnything) but make only limited use of 3D cues, restricting performance and generalization. Recently, visual geometry models such as VGGT have shown strong capability in providing rich 3D priors, but similar to monocular depth foundation models, they still operate at the level of visible surfaces rather than volumetric interiors, motivating us to explore how to more effectively leverage these increasingly powerful geometry priors for 3D occupancy prediction. We present GPOcc, a framework that leverages generalizable visual geometry priors (GPs) for monocular occupancy prediction. Our method extends surface points inward along camera rays to generate volumetric samples, which are represented as Gaussian primitives for probabilistic occupancy inference. To handle streaming input, we further design a training-free incremental update strategy that fuses per-frame Gaussians into a unified global representation. Experiments on Occ-ScanNet and EmbodiedOcc-ScanNet demonstrate significant gains: GPOcc improves mIoU by +9.99 in the monocular setting and +11.79 in the streaming setting over prior state of the art. Under the same depth prior, it achieves +6.73 mIoU while running 2.65$\times$ faster. These results highlight that GPOcc leverages geometry priors more effectively and efficiently. Code will be released at https://github.com/JuIvyy/GPOcc.

3D占据几何先验高斯表示单目重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。