提出垂直切片表征,提升室内3D语义占据预测精度
SliceOcc: Indoor 3D Semantic Occupancy Prediction with Vertical Slice Representation

- 用垂直切片替代平面视角,更好捕捉密集室内场景
- 在EmbodiedScan上达15.45% mIoU,刷新纯图像模型性能
- 适合机器人导航、AR/VR等需要精细室内理解的场景
3D语义占据预测是视觉感知的关键任务,需同时理解场景几何与语义信息,在机器人视觉和自动驾驶中有广泛应用。现有方法多采用俯视图(BEV)或三透视图(TPV)等平面表征,虽简化了3D复杂性,但在密集室内环境中因遮挡严重,难以有效捕获全局语义占据,导致性能下降。本文提出一种新的垂直切片表征,沿竖直方向分割场景,并将空间点特征投影到最近的一对平行平面上。基于此,构建了专用于室内3D语义占据预测的SliceOcc模型,利用切片查询与交叉注意力机制从图像中提取局部平面特征,再融合形成全局场景表示以完成占据预测。在EmbodiedScan数据集上的实验表明,该模型在81个室内类别上达到15.45%的mIoU,成为当前基于RGB相机的最优表现。代码已开源。
原文摘要 · Abstract (English)
3D semantic occupancy prediction is a crucial task in visual perception, as it requires the simultaneous comprehension of both scene geometry and semantics. It plays a crucial role in understanding 3D scenes and has great potential for various applications, such as robotic vision perception and autonomous driving. Many existing works utilize planar-based representations such as Bird's Eye View (BEV) and Tri-Perspective View (TPV). These representations aim to simplify the complexity of 3D scenes while preserving essential object information, thereby facilitating efficient scene representation. However, in dense indoor environments with prevalent occlusions, directly applying these planar-based methods often leads to difficulties in capturing global semantic occupancy, ultimately degrading model performance. In this paper, we present a new vertical slice representation that divides the scene along the vertical axis and projects spatial point features onto the nearest pair of parallel planes. To utilize these slice features, we propose SliceOcc, an RGB camera-based model specifically tailored for indoor 3D semantic occupancy prediction. SliceOcc utilizes pairs of slice queries and cross-attention mechanisms to extract planar features from input images. These local planar features are then fused to form a global scene representation, which is employed for indoor occupancy prediction. Experimental results on the EmbodiedScan dataset demonstrate that SliceOcc achieves a mIoU of 15.45% across 81 indoor categories, setting a new state-of-the-art performance among RGB camera-based models for indoor 3D semantic occupancy prediction. Code is available at https://github.com/NorthSummer/SliceOcc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。