arXiv:2507.04369cs.CV2025-07ICCV被引 9

用Mamba实现高效精准的多模态3D检测,兼顾速度与精度。

MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection

  • 基于Mamba块实现全局特征融合,线性复杂度提升效率。
  • 在nuScenes上达到75.0的NDS最优分数,超越高分辨率方法。
  • 通过高度保真编码解决对齐偏差,适合自动驾驶感知场景。

我们首次证明纯Mamba块可在保证顶尖性能的同时实现高效的密集全局融合,适用于相机-LiDAR多模态3D目标检测。现有融合策略受限于难以兼顾效率、长程建模和完整场景信息保留。受状态空间模型(SSMs)和线性注意力启发,我们利用其线性复杂度与长程建模能力应对挑战。但实验发现,简单采用低复杂度方法未必提升性能,甚至导致退化。原因在于多模态对齐过程中高度信息丢失,引发序列顺序偏移。为此,我们提出高度保真LiDAR编码,在连续空间中通过体素压缩保留精确高度信息,增强相机-LiDAR对齐。随后引入混合Mamba块,利用富含高度信息的特征进行局部与全局上下文学习。集成后,方法在nuScenes验证集上取得75.0的顶级NDS得分,优于使用高分辨率输入的方法,同时保持高效推理速度,显著快于多数最新先进方法。

原文摘要 · Abstract (English)

We present the first work demonstrating that a pure Mamba block can achieve efficient Dense Global Fusion, meanwhile guaranteeing top performance for camera-LiDAR multi-modal 3D object detection. Our motivation stems from the observation that existing fusion strategies are constrained by their inability to simultaneously achieve efficiency, long-range modeling, and retaining complete scene information. Inspired by recent advances in state-space models (SSMs) and linear attention, we leverage their linear complexity and long-range modeling capabilities to address these challenges. However, this is non-trivial since our experiments reveal that simply adopting efficient linear-complexity methods does not necessarily yield improvements and may even degrade performance. We attribute this degradation to the loss of height information during multi-modal alignment, leading to deviations in sequence order. To resolve this, we propose height-fidelity LiDAR encoding that preserves precise height information through voxel compression in continuous space, thereby enhancing camera-LiDAR alignment. Subsequently, we introduce the Hybrid Mamba Block, which leverages the enriched height-informed features to conduct local and global contextual learning. By integrating these components, our method achieves state-of-the-art performance with the top-tire NDS score of 75.0 on the nuScenes validation benchmark, even surpassing methods that utilize high-resolution inputs. Meanwhile, our method maintains efficiency, achieving faster inference speed than most recent state-of-the-art methods.

3D检测多模态融合Mamba自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。