用雷达点云和相机实现高效3D目标检测，精度与速度双突破。
PAN: Pillars-Attention-Based Network for 3D Object Detection
- 基于雷达支柱特征设计新骨干网络，引入自注意力建模点间关系。
- 在nuScenes数据集上达到58.2的NDS指标，刷新精度纪录。
- 简化卷积结构降低推理时间，适合实时系统部署。
摄像头与雷达融合为复杂天气和光照条件下的实时3D目标检测提供了一种鲁棒且低成本的替代方案。然而，现有研究中针对该模态的工作较少，尤其缺乏专门挖掘雷达点云优势（如精确测距、速度信息）的新架构。本文提出一种基于鸟瞰图（BEV）的新型高效3D目标检测算法，充分利用雷达特征在融合前的优势。设计新骨干网络，将雷达支柱特征映射至嵌入维度，并引入自注意力机制建模雷达点间的依赖关系。采用简化卷积层替代PointPillars类架构中的FPN结构，以显著降低推理时间。实验结果表明，该方法在nuScenes数据集上以ResNet-50为基础达到58.2的NDS指标，创下新纪录，同时在同类模型中设定最低推理时间基准。
原文摘要 · Abstract (English)
Camera-radar fusion offers a robust and low-cost alternative to Camera-lidar fusion for the 3D object detection task in real-time under adverse weather and lighting conditions. However, currently, in the literature, it is possible to find few works focusing on this modality and, most importantly, developing new architectures to explore the advantages of the radar point cloud, such as accurate distance estimation and speed information. Therefore, this work presents a novel and efficient 3D object detection algorithm using cameras and radars in the bird's-eye-view (BEV). Our algorithm exploits the advantages of radar before fusing the features into a detection head. A new backbone is introduced, which maps the radar pillar features into an embedded dimension. A self-attention mechanism allows the backbone to model the dependencies between the radar points. We are using a simplified convolutional layer to replace the FPN-based convolutional layers used in the PointPillars-based architectures with the main goal of reducing inference time. Our results show that with this modification, our approach achieves the new state-of-the-art in the 3D object detection problem, reaching 58.2 of the NDS metric for the use of ResNet-50, while also setting a new benchmark for inference time on the nuScenes dataset for the same category.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。