用激光雷达主导融合图像信息,提升3D检测精度与鲁棒性
BEVDilation: LiDAR-Centric Multi-Modal Fusion for 3D Object Detection
- 以激光雷达为中心,用图像特征做隐式引导而非直接拼接
- 在nuScenes上超越现有方法,且对深度噪声更鲁棒
- 适合追求高精度与稳定性的自动驾驶感知系统
将激光雷达与摄像头信息融合到鸟瞰图(BEV)表示中,在3D目标检测中已证明有效。然而,由于两类传感器在几何精度上的根本差异,以往方法的盲目融合常导致性能下降。本文提出BEVDilation,一种新型激光雷达主导的多模态融合框架。通过将图像BEV特征视为隐式引导而非简单拼接,有效缓解了由图像深度估计误差引起的空间错位问题。同时,图像引导有助于解决点云的稀疏性和语义局限性。具体地,提出稀疏体素膨胀模块,利用图像先验信息增强前景体素密度;引入语义引导的BEV膨胀模块,结合图像语义与长程上下文,提升特征扩散能力。在挑战性的nuScenes基准上,BEVDilation性能优于现有先进方法,同时保持高效计算。重要的是,该激光雷达主导策略对深度噪声更具鲁棒性。代码已开源。
原文摘要 · Abstract (English)
Integrating LiDAR and camera information in the bird's eye view (BEV) representation has demonstrated its effectiveness in 3D object detection. However, because of the fundamental disparity in geometric accuracy between these sensors, indiscriminate fusion in previous methods often leads to degraded performance. In this paper, we propose BEVDilation, a novel LiDAR-centric framework that prioritizes LiDAR information in the fusion. By formulating image BEV features as implicit guidance rather than naive concatenation, our strategy effectively alleviates the spatial misalignment caused by image depth estimation errors. Furthermore, the image guidance can effectively help the LiDAR-centric paradigm to address the sparsity and semantic limitations of point clouds. Specifically, we propose a Sparse Voxel Dilation Block that mitigates the inherent point sparsity by densifying foreground voxels through image priors. Moreover, we introduce a Semantic-Guided BEV Dilation Block to enhance the LiDAR feature diffusion processing with image semantic guidance and long-range context capture. On the challenging nuScenes benchmark, BEVDilation achieves better performance than state-of-the-art methods while maintaining competitive computational efficiency. Importantly, our LiDAR-centric strategy demonstrates greater robustness to depth noise compared to naive fusion. The source code is available at https://github.com/gwenzhang/BEVDilation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。