用相机与激光雷达互纠,提升3D多目标追踪的鲁棒性。
CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correction
- 两阶段设计:先粗生成轨迹,再通过跨模态修正精细优化。
- 在KITTI和nuScenes上显著优于18个基线方法,尤其在遮挡场景表现优异。
- 适合需要高精度实时追踪的自动驾驶系统开发者。
摄像头与激光雷达检测结果的融合为缓解3D多目标追踪中的跟踪失败提供了可行方案。然而,现有方法主要利用摄像头检测修正因激光雷达检测问题引发的跟踪错误,忽略了利用激光雷达数据优化摄像头检测的双向收益。这一局限源于其单阶段架构,类似单阶段目标检测器,缺乏专门的轨迹精修模块,无法充分挖掘多模态信息的互补优势。为此,我们提出CrossTracker,一种新型的在线多模态3D多目标追踪两阶段范式。CrossTracker采用由粗到精的策略,首先生成粗略轨迹,随后通过独立的精修过程进行优化。具体包含三个核心模块:i)多模态建模(M^3)模块,通过融合图像、点云甚至从图像中提取的平面几何信息,提供稳健的度量以支持后续轨迹生成;ii)粗轨迹生成(C-TG)模块,生成初始的双流轨迹;iii)轨迹精修(TR)模块,通过相机与激光雷达流之间的交叉修正来优化粗轨迹。全面实验表明,CrossTracker在性能上超越其18个竞争者,充分验证了其在利用摄像头与激光雷达传感器协同增益方面的能力。
原文摘要 · Abstract (English)
The fusion of camera- and LiDAR-based detections offers a promising solution to mitigate tracking failures in 3D multi-object tracking (MOT). However, existing methods predominantly exploit camera detections to correct tracking failures caused by potential LiDAR detection problems, neglecting the reciprocal benefit of refining camera detections using LiDAR data. This limitation is rooted in their single-stage architecture, akin to single-stage object detectors, lacking a dedicated trajectory refinement module to fully exploit the complementary multi-modal information. To this end, we introduce CrossTracker, a novel two-stage paradigm for online multi-modal 3D MOT. CrossTracker operates in a coarse-to-fine manner, initially generating coarse trajectories and subsequently refining them through an independent refinement process. Specifically, CrossTracker incorporates three essential modules: i) a multi-modal modeling (M^3) module that, by fusing multi-modal information (images, point clouds, and even plane geometry extracted from images), provides a robust metric for subsequent trajectory generation. ii) a coarse trajectory generation (C-TG) module that generates initial coarse dual-stream trajectories, and iii) a trajectory refinement (TR) module that refines coarse trajectories through cross correction between camera and LiDAR streams. Comprehensive experiments demonstrate the superior performance of our CrossTracker over its eighteen competitors, underscoring its effectiveness in harnessing the synergistic benefits of camera and LiDAR sensors for robust multi-modal 3D MOT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。