用深度信息提升多目标跟踪,解决遮挡和密集交互问题
DepTR-MOT: Unveiling the Potential of Depth-Informed Trajectory Refinement for Multi-Object Tracking
- 基于DETR框架融合实例级深度信息,改进轨迹预测
- 在QuadTrack和DanceTrack上分别取得27.59和44.47的HOTA分数
- 适合机器人场景中复杂遮挡环境下的多目标跟踪
视觉多目标跟踪是机器人感知的关键组件,现有基于检测的跟踪方法(TBD)主要依赖2D线索(如边界框和运动建模),在遮挡和近距离交互场景下表现不佳。尽管深度信息有潜力缓解这些问题,但多数现有MOT数据集缺乏深度标注,导致其应用受限。为此,本文提出DepTR-MOT,一种基于DETR的检测器,融合实例级深度信息。关键创新包括:(i) 基于基础模型的实例级软深度标签监督,提升深度预测精度;(ii) 将密集深度图蒸馏以保持全局深度一致性。该方法可在推理阶段输出实例级深度,无需额外基础模型或计算开销。引入深度线索显著提升了TBD范式的鲁棒性,有效解决遮挡与近距离交互挑战。在QuadTrack和DanceTrack数据集上的实验表明,该方法分别获得27.59和44.47的HOTA分数。尤其在机器人平台数据集QuadTrack上,展现出对遮挡和近距交互问题的优异处理能力。代码将开源。
原文摘要 · Abstract (English)
Visual Multi-Object Tracking (MOT) is a crucial component of robotic perception, yet existing Tracking-By-Detection (TBD) methods often rely on 2D cues, such as bounding boxes and motion modeling, which struggle under occlusions and close-proximity interactions. Trackers relying on these 2D cues are particularly unreliable in robotic environments, where dense targets and frequent occlusions are common. While depth information has the potential to alleviate these issues, most existing MOT datasets lack depth annotations, leading to its underexploited role in the domain. To unveil the potential of depth-informed trajectory refinement, we introduce DepTR-MOT, a DETR-based detector enhanced with instance-level depth information. Specifically, we propose two key innovations: (i) foundation model-based instance-level soft depth label supervision, which refines depth prediction, and (ii) the distillation of dense depth maps to maintain global depth consistency. These strategies enable DepTR-MOT to output instance-level depth during inference, without requiring foundation models and without additional computational cost. By incorporating depth cues, our method enhances the robustness of the TBD paradigm, effectively resolving occlusion and close-proximity challenges. Experiments on both the QuadTrack and DanceTrack datasets demonstrate the effectiveness of our approach, achieving HOTA scores of 27.59 and 44.47, respectively. In particular, results on QuadTrack, a robotic platform MOT dataset, highlight the advantages of our method in handling occlusion and close-proximity challenges in robotic tracking. The source code will be made publicly available at https://github.com/warriordby/DepTR-MOT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。