用深度图引导的全局查询网络，实现单目3D占位预测的高精度与低耗能。
DGOcc: Depth-aware Global Query-based Network for Monocular 3D Occupancy Prediction
- 基于深度图提取几何特征，通过全局查询模块融合图像与3D体素信息。
- 在SemanticKITTI和SSCBench-KITTI-360上达到最佳性能，内存占用更低。
- 适合自动驾驶等需高效3D场景理解的实时应用。
单目3D占位预测旨在仅从2D图像中预测3D场景中感兴趣区域的占据状态与语义，对3D场景理解具有重要意义。从2D图像预测大规模室外场景的3D占位存在病态且资源密集的问题。本文提出DGOcc——一种深度感知的全局查询网络，用于单目3D占位预测。首先利用先验深度图提取提供显式几何信息的深度上下文特征；随后设计全局查询（GQ）模块，通过注意力机制与尺度感知操作，实现图像与3D体素间的特征交互。此外，采用分层监督策略（HSS），避免将高维3D体素特征上采样至全分辨率，显著降低GPU内存占用与计算时间。在SemanticKITTI和SSCBench-KITTI-360数据集上的大量实验表明，该方法在单目语义占位预测中表现最优，同时大幅减少显存与运行时开销。
原文摘要 · Abstract (English)
Monocular 3D occupancy prediction, aiming to predict the occupancy and semantics within interesting regions of 3D scenes from only 2D images, has garnered increasing attention recently for its vital role in 3D scene understanding. Predicting the 3D occupancy of large-scale outdoor scenes from 2D images is ill-posed and resource-intensive. In this paper, we present \textbf{DGOcc}, a \textbf{D}epth-aware \textbf{G}lobal query-based network for monocular 3D \textbf{Occ}upancy prediction. We first explore prior depth maps to extract depth context features that provide explicit geometric information for the occupancy network. Then, in order to fully exploit the depth context features, we propose a Global Query-based (GQ) Module. The cooperation of attention mechanisms and scale-aware operations facilitates the feature interaction between images and 3D voxels. Moreover, a Hierarchical Supervision Strategy (HSS) is designed to avoid upsampling the high-dimension 3D voxel features to full resolution, which mitigates GPU memory utilization and time cost. Extensive experiments on SemanticKITTI and SSCBench-KITTI-360 datasets demonstrate that the proposed method achieves the best performance on monocular semantic occupancy prediction while reducing GPU and time overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。