用动态查询和状态空间模型,让多摄像头3D目标检测更快更准。
DySS: Dynamic Queries and State-Space Learning for Efficient 3D Object Detection from Multi-Camera Videos
- 通过状态空间模型动态处理时序特征,减少冗余计算。
- nuScenes测试集上达到65.31 NDS和57.4 mAP,优于现有方法。
- 适合需要实时推理的自动驾驶感知系统使用。
基于摄像头的鸟瞰图(BEV)3D目标检测是自动驾驶中关键的感知任务。早期方法依赖密集的BEV特征,构建成本高。近期工作虽采用稀疏查询检测,但仍需大量查询,视频帧数增多时计算开销上升。本文提出DySS,结合状态空间学习与动态查询机制。具体地,利用状态空间模型(SSM)对时序采样特征进行序列化处理,并引入未来预测与掩码重建作为辅助任务,以更好训练SSM,捕捉运动与对应关系。SSM的状态提供高效且信息丰富的场景表征。基于该表征,通过合并、删除、分裂操作动态更新检测查询,始终保持精炼有效的查询集合。DySS在保持高精度的同时实现高效推理:在nuScenes测试集上达到65.31 NDS和57.4 mAP,超越最新方法;在验证集上达56.2 NDS和46.2 mAP,且推理速度达33 FPS,满足实时性要求。
原文摘要 · Abstract (English)
Camera-based 3D object detection in Bird's Eye View (BEV) is one of the most important perception tasks in autonomous driving. Earlier methods rely on dense BEV features, which are costly to construct. More recent works explore sparse query-based detection. However, they still require a large number of queries and can become expensive to run when more video frames are used. In this paper, we propose DySS, a novel method that employs state-space learning and dynamic queries. More specifically, DySS leverages a state-space model (SSM) to sequentially process the sampled features over time steps. In order to encourage the model to better capture the underlying motion and correspondence information, we introduce auxiliary tasks of future prediction and masked reconstruction to better train the SSM. The state of the SSM then provides an informative yet efficient summarization of the scene. Based on the state-space learned features, we dynamically update the queries via merge, remove, and split operations, which help maintain a useful, lean set of detection queries throughout the network. Our proposed DySS achieves both superior detection performance and efficient inference. Specifically, on the nuScenes test split, DySS achieves 65.31 NDS and 57.4 mAP, outperforming the latest state of the art. On the val split, DySS achieves 56.2 NDS and 46.2 mAP, as well as a real-time inference speed of 33 FPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。