arXiv:2602.08126cs.CV2026-02被引 3

用状态空间模型融合相机与激光雷达,实现高效精准的3D目标检测。

MambaFusion: Adaptive State-Space Fusion for Multimodal 3D Object Detection

  • 采用可选状态空间模型与窗口化Transformer,线性时间传播全局上下文。
  • 在nuScenes上达到新SOTA,复杂度为线性,比传统方法更快更准。
  • 动态加权多模态特征,适合自动驾驶等需可靠感知的场景。

可靠的3D目标检测是自动驾驶的基础,而利用相机与激光雷达的多模态融合仍具挑战:相机提供密集视觉信息但深度不准确,激光雷达提供精确3D结构但覆盖稀疏。现有基于鸟瞰图(BEV)的融合框架存在上下文建模效率低、空间不变融合及不确定性推理不足等问题。本文提出MambaFusion,一个统一的多模态检测框架,实现高效、自适应且物理合理的3D感知。该框架将选择性状态空间模型(SSMs)与窗口化Transformer交错使用,在线性时间内传播全局上下文,同时保持局部几何保真度。多模态标记对齐(MTA)模块与可靠性感知融合门根据空间置信度和标定一致性动态重加权相机-激光雷达特征。最后,结构条件扩散头结合图推理与不确定性去噪,强化物理合理性与校准置信度。MambaFusion在nuScenes基准上达到新状态,且具有线性时间复杂度。结果表明,将基于SSM的高效性与可靠性驱动的融合相结合,可实现鲁棒、时序稳定且可解释的3D感知,适用于真实自动驾驶系统。

原文摘要 · Abstract (English)

Reliable 3D object detection is fundamental to autonomous driving, and multimodal fusion algorithms using cameras and LiDAR remain a persistent challenge. Cameras provide dense visual cues but ill posed depth; LiDAR provides a precise 3D structure but sparse coverage. Existing BEV-based fusion frameworks have made good progress, but they have difficulties including inefficient context modeling, spatially invariant fusion, and reasoning under uncertainty. We introduce MambaFusion, a unified multi-modal detection framework that achieves efficient, adaptive, and physically grounded 3D perception. MambaFusion interleaves selective state-space models (SSMs) with windowed transformers to propagate the global context in linear time while preserving local geometric fidelity. A multi-modal token alignment (MTA) module and reliability-aware fusion gates dynamically re-weight camera-LiDAR features based on spatial confidence and calibration consistency. Finally, a structure-conditioned diffusion head integrates graph-based reasoning with uncertainty-aware denoising, enforcing physical plausibility, and calibrated confidence. MambaFusion establishes new state-of-the-art performance on nuScenes benchmarks while operating with linear-time complexity. The framework demonstrates that coupling SSM-based efficiency with reliability-driven fusion yields robust, temporally stable, and interpretable 3D perception for real-world autonomous driving systems.

3D检测多模态融合状态空间模型自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。