融合摄像头与激光雷达，提升3D目标检测精度。
A Multimodal Hybrid Late-Cascade Fusion Network for Enhanced 3D Object Detection
- 先分别用相机和激光雷达检测，再通过投影匹配融合结果。
- 减少激光雷达误检，恢复漏检目标，对行人和骑行者效果显著。
- 可对接现有检测器，灵活训练，适合自动驾驶场景。
我们提出一种基于多模态输入的3D目标检测新方法，采用混合式后期级联融合架构，结合RGB相机检测网络与3D激光雷达（LiDAR）探测器。通过将激光雷达检测框投影至图像，实现与视觉检测结果的匹配，以降低激光雷达误报率；利用视图间极线约束和由多视角相机检测生成的视锥体，恢复激光雷达漏检目标。该方案可无缝集成到任意单模态检测器之上，支持预训练模型复用或分阶段独立训练。在KITTI数据集上的实验表明，该方法在行人和骑行者检测上取得显著性能提升。
原文摘要 · Abstract (English)
We present a new way to detect 3D objects from multimodal inputs, leveraging both LiDAR and RGB cameras in a hybrid late-cascade scheme, that combines an RGB detection network and a 3D LiDAR detector. We exploit late fusion principles to reduce LiDAR False Positives, matching LiDAR detections with RGB ones by projecting the LiDAR bounding boxes on the image. We rely on cascade fusion principles to recover LiDAR False Negatives leveraging epipolar constraints and frustums generated by RGB detections of separate views. Our solution can be plugged on top of any underlying single-modal detectors, enabling a flexible training process that can take advantage of pre-trained LiDAR and RGB detectors, or train the two branches separately. We evaluate our results on the KITTI object detection benchmark, showing significant performance improvements, especially for the detection of Pedestrians and Cyclists.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。