arXiv:2503.14493cs.CVcs.AI2025-03ICLR被引 9

用状态空间模型重构3D目标检测,实现线性复杂度的特征更新。

State Space Model Meets Transformer: A New Paradigm for 3D Object Detection

  • 将查询作为系统状态,点云作为输入,实现双向交互式特征更新。
  • 在ScanNetV2和SUN RGB-D上分别提升AP50 5.3和3.2,刷新性能记录。
  • 适合追求高效高精度3D检测的工业与自动驾驶场景。

基于DETR的方法通过多层Transformer解码器迭代优化目标查询,在3D室内目标检测中表现优异。然而,解码器中的场景点特征保持固定,导致后期解码层贡献微弱,限制了性能提升。最近,状态空间模型(SSM)通过系统状态与输入间的迭代交互,以线性复杂度展现出高效的上下文建模能力。受此启发,我们提出一种新的3D目标检测范式——交互式状态空间模型(DEST)。在该模型中,设计了一种新颖的状态依赖参数化方法,使系统状态能有效作为3D室内检测任务中的查询。此外,针对点云与SSM特性,引入四项关键设计:序列化与双向扫描策略实现点间双向特征交互;状态间注意力机制建模状态点间关系;门控前馈网络增强通道间相关性。据我们所知,这是首个将查询建模为系统状态、场景点作为系统输入的方法,可在线性复杂度下同步更新场景点特征与查询特征。在两个挑战性数据集上的大量实验表明,基于DEST的方法显著有效:相较GroupFree基线,在ScanNet V2上提升AP50 5.3,SUN RGB-D上提升3.2;基于VDETR基线,在ScanNetV2和SUN RGB-D上均达到新最优(SOTA)。

原文摘要 · Abstract (English)

DETR-based methods, which use multi-layer transformer decoders to refine object queries iteratively, have shown promising performance in 3D indoor object detection. However, the scene point features in the transformer decoder remain fixed, leading to minimal contributions from later decoder layers, thereby limiting performance improvement. Recently, State Space Models (SSM) have shown efficient context modeling ability with linear complexity through iterative interactions between system states and inputs. Inspired by SSMs, we propose a new 3D object DEtection paradigm with an interactive STate space model (DEST). In the interactive SSM, we design a novel state-dependent SSM parameterization method that enables system states to effectively serve as queries in 3D indoor detection tasks. In addition, we introduce four key designs tailored to the characteristics of point cloud and SSM: The serialization and bidirectional scanning strategies enable bidirectional feature interaction among scene points within the SSM. The inter-state attention mechanism models the relationships between state points, while the gated feed-forward network enhances inter-channel correlations. To the best of our knowledge, this is the first method to model queries as system states and scene points as system inputs, which can simultaneously update scene point features and query features with linear complexity. Extensive experiments on two challenging datasets demonstrate the effectiveness of our DEST-based method. Our method improves the GroupFree baseline in terms of AP50 on ScanNet V2 (+5.3) and SUN RGB-D (+3.2) datasets. Based on the VDETR baseline, Our method sets a new SOTA on the ScanNetV2 and SUN RGB-D datasets.

3D检测状态空间Transformer点云

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。