用3D物体检测辅助，提升自动驾驶环境语义占位预测精度。
Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection
- 引入3D物体检测作为辅助任务，增强中间特征表达能力。
- 在nuScenes数据集上达到31.73%的IoU和20.91%的mIoU，领先当前方法。
- 特别擅长识别行人、自行车等易忽略的脆弱道路使用者。
3D语义占位预测旨在利用车载环视摄像头为自动驾驶车辆预测周围环境的详细几何与语义信息。现有方法主要聚焦于复杂内部结构模块设计，如高效特征采样与聚合流程或中间特征表示形式。本文通过多任务学习，引入3D物体检测作为辅助分支，提供额外的3D监督信号。该信号增强了模型对场景中小型动态物体的捕捉能力，而这些物体通常包括行人、自行车和摩托车等脆弱道路使用者（VRU），其检测对行车安全至关重要。在nuScenes数据集上的大量实验，涵盖雨天和夜间等挑战性场景，表明本方法达到当前最优性能，实现31.73%的IoU与20.91%的mIoU，并显著提升对脆弱道路使用者的检测效果。
原文摘要 · Abstract (English)
3D semantic occupancy prediction aims to forecast detailed geometric and semantic information of the surrounding environment for autonomous vehicles (AVs) using onboard surround-view cameras. Existing methods primarily focus on intricate inner structure module designs to improve model performance, such as efficient feature sampling and aggregation processes or intermediate feature representation formats. In this paper, we explore multitask learning by introducing an additional 3D supervision signal by incorporating an additional 3D object detection auxiliary branch. This extra 3D supervision signal enhances the model's overall performance by strengthening the capability of the intermediate features to capture small dynamic objects in the scene, and these small dynamic objects often include vulnerable road users, i.e. bicycles, motorcycles, and pedestrians, whose detection is crucial for ensuring driving safety in autonomous vehicles. Extensive experiments conducted on the nuScenes datasets, including challenging rainy and nighttime scenarios, showcase that our approach attains state-of-the-art results, achieving an IoU score of 31.73% and a mIoU score of 20.91% and excels at detecting vulnerable road users (VRU). The code will be made available at:https://github.com/DanielMing123/Inverse++
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。