用Transformer与状态空间模型结合,提升点云中远距离小物体检测精度。
3DTMDet: A Dual-Path Synergy Network of Transformer and SSM for 3D Object Detection in Point Clouds

- 融合Mamba与Transformer,分治全局上下文与局部几何细节。
- 在KITTI和ONCE数据集上优于现有方法,远距离小目标检测显著提升。
- 适合关注点云3D检测、自动驾驶感知的研究者与工程师。
点云目标检测的核心挑战在于远距离点云极度稀疏与远距离上下文理解需求之间的矛盾。现有方法通常采用1D序列化扩展感受野,但会丢失本已稀缺的局部几何细节,降低对远距离和小目标的检测能力。为此,我们提出3DTMDet,一种将状态空间模型(Mamba)与Transformer协同结合的新检测网络。核心思想是利用SSM线性复杂度和长序列建模优势,有效捕捉稀疏远点间的全局交互;同时使用具有局部注意力的Transformer模块,编码局部点集中的精细几何结构,保留准确形状信息。我们设计了3D混合Mamba-Transformer(3DHMT)模块,采用SSM-Attention-SSM流水线,在扩大感受野的同时保持局部细节。此外,受激光雷达物理特性启发,引入体素生成模块,沿传感器观测方向扩散特征,重建遮挡及远距离区域的完整物体结构。在KITTI和ONCE数据集上的大量实验表明,3DTMDet性能超越现有最先进检测器。代码已开源:https://github.com/QiuBingwen/3DTMDet。
原文摘要 · Abstract (English)
A fundamental challenge in point cloud object detection lies in the conflict between the extreme sparsity of distant points and the need for remote context understanding. The existing methods typically use 1D serialization to expand the receptive field, which inevitably discards already scarce local geometric details and reduces detection of distant and small objects. To address this issue, we propose 3DTMDet, a novel detection network that synergistically combines state space models (Mamba) with Transformers. The core idea is to utilize SSM's linear complexity and advantages in long sequence modeling to effectively capture global interactions between sparse and distant points, while using Transformer modules with local attention to encode fine-grained geometric structures in local point sets, preserving accurate shape information. We propose the 3D Hybrid Mamba Transformer (3DHMT) block, which uses an SSM-Attention-SSM pipeline to balance global context understanding and local detail preservation, effectively alleviating the tension between receptive field enlargement and geometric preservation in remote detection. In addition, we introduced a voxel generation block inspired by LiDAR physics, which diffuses features along the sensor observation direction to reconstruct the complete object structure of occlusion and distant areas. Extensive experiments conducted on the KITTI and ONCE datasets have shown that 3DTMDet outperforms state-of-the-art detectors. The code is available at https://github.com/QiuBingwen/3DTMDet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。