用新模型提升单目3D多目标跟踪精度与速度,效果领先。
S3MOT: Monocular 3D Object Tracking with Selective State Space Model
- 设计新型状态空间模型实现高效全局关联决策。
- 在KITTI上达76.86 HOTA,速度31 FPS,性能领先。
- 适合做自动驾驶、机器人感知的开发者参考。
单目3D多目标跟踪对机器人和计算机视觉应用至关重要,但因从2D视频流中挖掘3D时空关联困难而极具挑战。本文提出三项创新技术:(1) 匈牙利状态空间模型(HSSM),通过跨路径压缩上下文线索,以线性复杂度实现高效全面的关联决策,具备全局感受野和动态权重;(2) 全卷积单阶段嵌入(FCOE),摒弃区域建议池化,直接使用密集特征图进行对比学习,在视角和光照变化下显著提升重识别准确率;(3) 速度状态空间模型(VeloSSM),采用编码器-解码器结构建模速度时序依赖,捕捉运动动态,克服帧级3D推理局限。在KITTI公开测试集上,本方法达到76.86 HOTA,推理速度为31 FPS,相比前人最优结果提升+2.63 HOTA和+3.62 AssA,充分验证了其鲁棒性与高效性。代码与模型已开源。
原文摘要 · Abstract (English)
Accurate and reliable multi-object tracking (MOT) in 3D space is essential for advancing robotics and computer vision applications. However, it remains a significant challenge in monocular setups due to the difficulty of mining 3D spatiotemporal associations from 2D video streams. In this work, we present three innovative techniques to enhance the fusion and exploitation of heterogeneous cues for monocular 3D MOT: (1) we introduce the Hungarian State Space Model (HSSM), a novel data association mechanism that compresses contextual tracking cues across multiple paths, enabling efficient and comprehensive assignment decisions with linear complexity. HSSM features a global receptive field and dynamic weights, in contrast to traditional linear assignment algorithms that rely on hand-crafted association costs. (2) We propose Fully Convolutional One-stage Embedding (FCOE), which eliminates ROI pooling by directly using dense feature maps for contrastive learning, thus improving object re-identification accuracy under challenging conditions such as varying viewpoints and lighting. (3) We enhance 6-DoF pose estimation through VeloSSM, an encoder-decoder architecture that models temporal dependencies in velocity to capture motion dynamics, overcoming the limitations of frame-based 3D inference. Experiments on the KITTI public test benchmark demonstrate the effectiveness of our method, achieving a new state-of-the-art performance of 76.86~HOTA at 31~FPS. Our approach outperforms the previous best by significant margins of +2.63~HOTA and +3.62~AssA, showcasing its robustness and efficiency for monocular 3D MOT tasks. The code and models are available at https://github.com/bytepioneerX/s3mot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。