通过强化视觉信息利用，显著提升自动驾驶3D目标检测性能。
Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitation
- 设计对比对齐模块与相机辅助分支，充分挖掘图像特征
- 仅用6轮训练达69.7% mAP，参数增益仅3.48%
- 适合追求高效高精度检测的自动驾驶研究者
当前多数方法在鸟瞰图空间融合激光雷达与图像数据以实现自动驾驶中的3D目标检测。然而，现有方法过度依赖激光雷达分支，对图像信息利用不足。为此，我们提出Fusion4CA，基于经典BEVFusion框架，通过即插即用组件全面挖掘视觉输入。具体包括：设计对比对齐模块，将图像特征与三维几何信息对齐；引入相机辅助分支，在训练中充分挖掘RGB信息；利用预训练图像权重的认知适配器，最大化利用预训练模型；在融合阶段集成标准坐标注意力模块作为补充增强。在nuScenes数据集上的实验表明，该方法仅需6个训练周期即可达到69.7% mAP，推理参数仅增加3.48%，相比完全训练20轮的基线提升1.2%。在模拟月球环境中的大量实验进一步验证了方法的有效性与泛化能力。代码将随Fusion4CA开源。
原文摘要 · Abstract (English)
Nowadays, an increasing number of works fuse LiDAR and RGB data in the bird's-eye view (BEV) space for 3D object detection in autonomous driving systems. However, existing methods suffer from over-reliance on the LiDAR branch, with insufficient exploration of RGB information. To tackle this issue, we propose Fusion4CA, which is built upon the classic BEVFusion framework and dedicated to fully exploiting visual input with plug-and-play components. Specifically, a contrastive alignment module is designed to calibrate image features with 3D geometry, and a camera auxiliary branch is introduced to mine RGB information sufficiently during training. For further performance enhancement, we leverage an off-the-shelf cognitive adapter to make the most of pretrained image weights, and integrate a standard coordinate attention module into the fusion stage as a supplementary boost. Experiments on the nuScenes dataset demonstrate that our method achieves 69.7% mAP with only 6 training epochs and a mere 3.48% increase in inference parameters, yielding a 1.2% improvement over the baseline which is fully trained for 20 epochs. Extensive experiments in a simulated lunar environment further validate the effectiveness and generalization of our method. Our code will be released through Fusion4CA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。