通过增强雷达特征密度与质量，提升相机-雷达融合的3D占位预测性能。
REOcc: Camera-Radar Fusion with Radar Feature Enrichment for 3D Occupancy Prediction
- 设计雷达稠密化与增强模块，融合空间和上下文信息优化雷达特征。
- 在Occ3D-nuScenes上相比纯相机模型，动态物体类别的占位预测精度显著提升。
- 适合需要高鲁棒性3D环境感知的自动驾驶系统使用。
基于视觉的3D占位预测已取得显著进展，但仅依赖摄像头在复杂环境下表现受限。为此，相机-雷达融合因两者互补优势成为有前景的解决方案。然而，雷达数据的稀疏性和噪声限制了其效果，导致融合性能不佳。本文提出REOcc，一种新型相机-雷达融合网络，旨在通过增强雷达特征表示来提升3D占位预测性能。方法引入两个核心组件：雷达稠密化模块（Radar Densifier）和雷达增强模块（Radar Amplifier），通过整合空间与上下文信息，有效提升雷达特征的空间密度与质量。在Occ3D-nuScenes基准上的大量实验表明，相比纯相机基线模型，REOcc在动态物体类别上实现显著性能提升，验证了其缓解雷达数据稀疏与噪声的能力。由此，雷达可更有效地补充摄像头数据，充分释放相机-雷达融合在鲁棒、可靠的3D占位预测中的潜力。
原文摘要 · Abstract (English)
Vision-based 3D occupancy prediction has made significant advancements, but its reliance on cameras alone struggles in challenging environments. This limitation has driven the adoption of sensor fusion, among which camera-radar fusion stands out as a promising solution due to their complementary strengths. However, the sparsity and noise of the radar data limits its effectiveness, leading to suboptimal fusion performance. In this paper, we propose REOcc, a novel camera-radar fusion network designed to enrich radar feature representations for 3D occupancy prediction. Our approach introduces two main components, a Radar Densifier and a Radar Amplifier, which refine radar features by integrating spatial and contextual information, effectively enhancing spatial density and quality. Extensive experiments on the Occ3D-nuScenes benchmark demonstrate that REOcc achieves significant performance gains over the camera-only baseline model, particularly in dynamic object classes. These results underscore REOcc's capability to mitigate the sparsity and noise of the radar data. Consequently, radar complements camera data more effectively, unlocking the full potential of camera-radar fusion for robust and reliable 3D occupancy prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。