轻量级空间嵌入提升3D占位预测效率,兼顾精度与部署速度。
LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Prediction

- 用单通道深度采样结合2D卷积生成紧凑空间嵌入,替代传统体素表示。
- 在多个基准上达到顶尖性能,推理速度比主流方法快3倍以上。
- 适合自动驾驶等需实时3D环境感知的边缘设备部署。
近年来,占位预测因其精细的环境表征和对开放集物体的良好泛化能力而受到关注。然而,主流方法使用复杂的体素特征作为场景表示,带来巨大的内存和计算开销。我们发现鸟瞰图(BEV)平面中高度维度的信息熵远低于另外两个维度,表明其占位分布更易学习和预测。为此,我们提出轻量级空间嵌入(Lightweight Spatial Embedding),以更紧凑的方式表征完整高度信息,显著提升可部署性。首先,从多视角深度分布中采样单通道占位,通过空间转通道机制利用2D卷积提取各视角的轻量嵌入;再通过轻量跨视角交互模块融合为统一嵌入,直接补充BEV特征的高度信息。此外,引入边缘感知嵌入并施加几何监督,增强空间表征能力。我们还提出基于特征层面的BEV-CutMix数据增强策略,提升驾驶场景多样性。上述组件集成于纯2D卷积模型LightOcc,实验表明其在多个基准上达到当前最优性能,同时展现显著效率优势。
原文摘要 · Abstract (English)
Occupancy prediction has garnered increasing attention in recent years for its comprehensive fine-grained environmental representation and strong generalization to open-set objects. Nevertheless, mainstream occupancy prediction methods employ cumbersome voxel features as the scene representation, incurring substantial overheads in both memory and computation. When comparing the occupancy distribution in each spatial dimension, we find that the information entropy of the height dimension is much lower than the other two dimensions that constitute the Bird's Eye View (BEV) plane, which indicates that the height distribution of occupancy is easier to learn and predict. Accordingly, we propose Lightweight Spatial Embedding that can represent complete height information in a more compact way than voxel features, thus significantly enhancing its deployability. First, Single-Channel Occupancy is sampled from the multi-view depth distributions, which is then processed by Spatial-to-Channel mechanism to extract Lightweight Spatial Embeddings of different views by 2D convolution. These embeddings will interact with each other through the Lightweight Cross-View Interaction module to obtain the Unified Embedding, which can directly supplement BEV features with height information. Furthermore, we extract Edge-aware Spatial Embedding and apply Geometric Supervision on Spatial Embeddings, aiming to enhance their capability to represent spatial information. We also propose BEV-CutMix, a feature-level data augmentation strategy, to increase the diversity of the driving scenes. We integrate these innovative components into a pure 2D convolutional model, namely LightOcc. Sufficient experimental results show that LightOcc achieves state-of-the-art performance on multiple benchmarks while demonstrating significant efficiency advantages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。