通过显式高度解耦提升视觉3D占据预测精度
Deep Height Decoupling for Precise Vision-based 3D Occupancy Prediction
- 引入显式高度先验，分离不同高度层的特征
- 在nuScenes数据集上用极少输入帧达领先性能
- 适合自动驾驶中3D环境重建场景
基于视觉的3D占据预测旨在从2D彩色图像重建3D几何结构并估计语义类别，其中2D到3D的视角转换是关键步骤。现有方法如BEVPooling和VoxelPooling采用前向投影将2D图像特征映射到3D网格，但同一高度范围内的网格常混入其他高度的干扰特征。为此，我们提出深度高度解耦（DHD）框架，通过显式监督预测高度图，并基于高度分布统计设计掩码引导高度采样（MGHS），将高度图自适应分解为多个二值掩码，将2D特征投影至多个子空间，每个网格仅保留合理高度范围内的特征。最后，通过协同特征聚合（SFA）模块利用通道与空间亲和性增强特征表示，实现更优占据预测。在主流Occ3D-nuScenes基准上，本方法即使使用最少输入帧也达到当前最优性能。源代码已开源：https://github.com/yanzq95/DHD。
原文摘要 · Abstract (English)
The task of vision-based 3D occupancy prediction aims to reconstruct 3D geometry and estimate its semantic classes from 2D color images, where the 2D-to-3D view transformation is an indispensable step. Most previous methods conduct forward projection, such as BEVPooling and VoxelPooling, both of which map the 2D image features into 3D grids. However, the current grid representing features within a certain height range usually introduces many confusing features that belong to other height ranges. To address this challenge, we present Deep Height Decoupling (DHD), a novel framework that incorporates explicit height prior to filter out the confusing features. Specifically, DHD first predicts height maps via explicit supervision. Based on the height distribution statistics, DHD designs Mask Guided Height Sampling (MGHS) to adaptively decouple the height map into multiple binary masks. MGHS projects the 2D image features into multiple subspaces, where each grid contains features within reasonable height ranges. Finally, a Synergistic Feature Aggregation (SFA) module is deployed to enhance the feature representation through channel and spatial affinities, enabling further occupancy refinement. On the popular Occ3D-nuScenes benchmark, our method achieves state-of-the-art performance even with minimal input frames. Source code is released at https://github.com/yanzq95/DHD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。