用深度和掩码特征自监督学习，提升遮挡下多目标跟踪精度。
Fast Self-Supervised depth and mask aware Association for Multi-Object Tracking
- 融合深度与掩码特征，通过自监督编码器生成稳定表征。
- 在SportsMOT、DanceTrack等复杂场景上超越现有TBD方法。
- 无需计算掩码IoU，适合高密度遮挡场景的实时跟踪应用。
多目标跟踪（MOT）通常依赖交并比（IoU）进行关联，但在目标相似或遮挡时不可靠，且计算分割掩码的IoU代价高昂。本文利用分割掩码捕捉物体形状，但不计算掩码交并比。我们融合深度与掩码特征，输入一个自监督训练的轻量编码器，生成稳定的对象表征，作为边界框IoU和重识别特征之外的额外相似性线索。深度图由零样本深度估计算器获得，掩码由可提示视觉分割模型生成，以获取细粒度空间信息。我们的方法是首个在不计算掩码IoU的前提下，使用自监督编码器优化分割掩码的TBD式MOT方案。在包含非线性运动、遮挡和密集场景的挑战性基准（如SportsMOT和DanceTrack）上，该方法在多数指标上优于当前最先进的TBD模型；在简单线性运动场景（如MOT17）上也保持竞争力。
原文摘要 · Abstract (English)
Multi-object tracking (MOT) methods often rely on Intersection-over-Union (IoU) for association. However, this becomes unreliable when objects are similar or occluded. Also, computing IoU for segmentation masks is computationally expensive. In this work, we use segmentation masks to capture object shapes, but we do not compute segmentation IoU. Instead, we fuse depth and mask features and pass them through a compact encoder trained self-supervised. This encoder produces stable object representations, which we use as an additional similarity cue alongside bounding box IoU and re-identification features for matching. We obtain depth maps from a zero-shot depth estimator and object masks from a promptable visual segmentation model to obtain fine-grained spatial cues. Our MOT method is the first to use the self-supervised encoder to refine segmentation masks without computing masks IoU. MOT can be divided into joint detection-ReID (JDR) and tracking-by-detection (TBD) models. The latter are computationally more efficient. Experiments of our TBD method on challenging benchmarks with non-linear motion, occlusion, and crowded scenes, such as SportsMOT and DanceTrack, show that our method outperforms the TBD state-of-the-art on most metrics, while achieving competitive performance on simpler benchmarks with linear motion, such as MOT17.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。