用3D目标检测辅助多传感器融合，提升占用预测精度与部署效率。
DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction
- 以3D目标检测作为监督信号，指导多模态特征融合
- 仅用ResNet-50和256×704输入，实现新最佳性能
- 支持高效部署，在RTX 4090上达104.9 FPS
多传感器融合显著提升了3D语义占用预测的准确性和鲁棒性，对自动驾驶与机器人至关重要。然而，现有方法依赖高分辨率图像和复杂网络，限制了实际部署。同时，多数融合策略只关注特征融合，忽视了有效的监督机制。为此，我们提出DAOcc，一种新型多模态占用预测框架，利用3D目标检测监督来提升性能，同时采用轻量级图像主干和实用输入分辨率。此外，引入BEV视图范围扩展策略，缓解低分辨率带来的性能下降。大量实验表明，DAOcc在Occ3D-nuScenes和Occ3D-Waymo两个基准上均达到新SOTA，仅使用ResNet-50和256×704输入即显著超越此前最优方法。经TensorRT优化后，可在NVIDIA RTX 4090上实现104.9 FPS，mIoU保持54.2。代码已开源。
原文摘要 · Abstract (English)
Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。