arXiv:2602.22667cs.CV2026-02中稿 · CVPR被引 5

用纯几何监督实现室内场景开放词汇占位预测,效果超越现有方法。

Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes

  • 基于3D语言嵌入高斯点,统一表示几何与语义信息。
  • 在Occ-ScanNet上达59.50的IoU和21.05的mIoU,显著领先。
  • 适合需要理解复杂室内语义的机器人导航任务。

开放词汇3D占位对具身智能体至关重要,因其需理解语义丰富且动态演化的复杂室内环境。尽管已有研究探索了室外驾驶场景中的开放词汇占位,但这些方法在室内表现不佳,因室内几何更密集、布局更复杂、语义更细粒度。为此,本文采用仅含二值占位标签(占据/空闲)的纯几何监督范式。框架基于3D语言嵌入高斯点,作为耦合细粒度几何与语言对齐语义嵌入的统一中间表示。在几何方面,发现现有高斯到占位映射器在弱监督下无法收敛,提出一种考虑透明度的泊松聚合方法以稳定体积融合。在语义方面,渲染特征与开放词汇分割特征直接对齐存在特征混合问题,因此设计渐进温度衰减策略,在投射过程中逐步锐化透明度,增强高斯与语言的对齐。在Occ-ScanNet数据集上,本方法在开放词汇设置下取得59.50的IoU和21.05的mIoU,所有现有占位方法中最高,且大幅超越以往开放词汇方法。代码将发布于https://github.com/JuIvyy/LegoOcc。

原文摘要 · Abstract (English)

Open-vocabulary 3D occupancy is vital for embodied agents, which need to understand complex indoor environments where semantic categories are abundant and evolve beyond fixed taxonomies. While recent work has explored open-vocabulary occupancy in outdoor driving scenarios, such methods transfer poorly indoors, where geometry is denser, layouts are more intricate, and semantics are far more fine-grained. To address these challenges, we adopt a geometry-only supervision paradigm that uses only binary occupancy labels (occupied vs free). Our framework builds upon 3D Language-Embedded Gaussians, which serve as a unified intermediate representation coupling fine-grained 3D geometry with a language-aligned semantic embedding. On the geometry side, we find that existing Gaussian-to-Occupancy operators fail to converge under such weak supervision, and we introduce an opacity-aware, Poisson-based approach that stabilizes volumetric aggregation. On the semantic side, direct alignment between rendered features and open-vocabulary segmentation features suffers from feature mixing; we therefore propose a Progressive Temperature Decay schedule that gradually sharpens opacities during splatting, strengthening Gaussian-language alignment. On Occ-ScanNet, our framework achieves 59.50 IoU and 21.05 mIoU in the open-vocabulary setting, surpassing all existing occupancy methods in IoU and outperforming prior open-vocabulary approaches by a large margin in mIoU. Code will be released at https://github.com/JuIvyy/LegoOcc.

3D占位开放词汇室内建模高斯溅射

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。