基于农机道路场景的三维目标检测新网络，融合图像与点云信息提升精度。
FrustumFusionNets: A Three-Dimensional Object Detection Network Based on Tractor Road Scene
- 用图像检测结果限定点云搜索区域，构建双模态特征提取框架。
- 在农机道路数据集上，对车和人检测准确率分别达82.28%和95.68%，优于原模型1.83%和2.33%。
- 适合无人农机在复杂道路场景中进行高精度实时三维目标检测。
针对现有基于视锥的方法在道路三维目标检测中图像信息利用不足以及农业场景研究缺乏的问题，本文在复杂农机道路场景下，结合80线激光雷达与摄像头构建了三维目标检测数据集，并提出FrustumFusionNets（FFNets）新网络。首先，利用图像二维目标检测结果缩小点云三维空间的搜索范围；其次，引入高斯掩码增强点云信息；然后，分别通过点云与图像特征提取管道获取特征；最后，融合双模态特征实现三维目标检测。实验表明，在自建农机道路测试集上，FrustumFusionNetv2对车辆和行人的三维检测准确率分别为82.28%和95.68%，较原模型提升1.83%和2.33%。该方法在KITTI基准测试集上也展现出对行人检测的显著优势，为无人农机在农机道路场景下的多目标、高精度、实时三维检测提供了有效方案。
原文摘要 · Abstract (English)
To address the issues of the existing frustum-based methods' underutilization of image information in road three-dimensional object detection as well as the lack of research on agricultural scenes, we constructed an object detection dataset using an 80-line Light Detection And Ranging (LiDAR) and a camera in a complex tractor road scene and proposed a new network called FrustumFusionNets (FFNets). Initially, we utilize the results of image-based two-dimensional object detection to narrow down the search region in the three-dimensional space of the point cloud. Next, we introduce a Gaussian mask to enhance the point cloud information. Then, we extract the features from the frustum point cloud and the crop image using the point cloud feature extraction pipeline and the image feature extraction pipeline, respectively. Finally, we concatenate and fuse the data features from both modalities to achieve three-dimensional object detection. Experiments demonstrate that on the constructed test set of tractor road data, the FrustumFusionNetv2 achieves 82.28% and 95.68% accuracy in the three-dimensional object detection of the two main road objects, cars and people, respectively. This performance is 1.83% and 2.33% better than the original model. It offers a hybrid fusion-based multi-object, high-precision, real-time three-dimensional object detection technique for unmanned agricultural machines in tractor road scenarios. On the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) Benchmark Suite validation set, the FrustumFusionNetv2 also demonstrates significant superiority in detecting road pedestrian objects compared with other frustum-based three-dimensional object detection methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。