用文字提示追踪没见过的物体,无需训练样本。
TP-GMOT: Tracking Generic Multiple Object by Textual Prompt with Motion-Appearance Cost (MAC) SORT
- 通过文本提示实现零样本检测未知类别物体。
- 融合运动与外观特征,提升高相似物体的追踪准确率。
- 适用于无预设类别的通用多目标追踪场景。
尽管多目标追踪(MOT)已取得显著进展,但其严重依赖先验知识,且仅限于预定义类别。相比之下,通用多目标追踪(GMOT)可追踪外观相似的多个物体,对目标先验信息要求较低,但面临视角、光照、遮挡和分辨率变化等挑战。本文提出首个面向文本描述的通用多目标追踪数据集「Refer-GMOT」,并设计基于文本提示的开放词汇GMOT框架TP-GMOT,可在零训练样本条件下追踪未见过的物体类别。该框架包含两项创新:(i) TP-OD,一种基于文本提示的目标检测方法,可精准定位具有特定属性的未知物体;(ii) Motion-Appearance Cost SORT(MAC-SORT),一种融合运动与外观匹配策略的新关联方法,有效应对高相似度物体的追踪难题。我们在Refer-GMOT数据集上评估了该框架在GMOT任务中的表现,并在DanceTrack与MOT20数据集上进行消融实验,验证了TP-GMOT框架的泛化能力与MAC-SORT的有效性。代码与模型将公开发布。
原文摘要 · Abstract (English)
While Multi-Object Tracking (MOT) has made substantial advancements, it is limited by heavy reliance on prior knowledge and limited to predefined categories. In contrast, Generic Multiple Object Tracking (GMOT), tracking multiple objects with similar appearance, requires less prior information about the targets but faces challenges with variants like viewpoint, lighting, occlusion, and resolution. Our contributions commence with the introduction of the \textbf{\text{Refer-GMOT dataset}} a collection of videos, each accompanied by fine-grained textual descriptions of their attributes. Subsequently, we introduce a novel text prompt-based open-vocabulary GMOT framework, called \textbf{\text{TP-GMOT}}, which can track never-seen object categories with zero training examples. Within \text{TP-GMOT} framework, we introduce two novel components: (i) {\textbf{\text{TP-OD}}, an object detection by a textual prompt}, for accurately detecting unseen objects with specific characteristics. (ii) Motion-Appearance Cost SORT \textbf{\text{MAC-SORT}}, a novel object association approach that adeptly integrates motion and appearance-based matching strategies to tackle the complex task of tracking multiple generic objects with high similarity. Our contributions are benchmarked on the \text{Refer-GMOT} dataset for GMOT task. Additionally, to assess the generalizability of the proposed \text{TP-GMOT} framework and the effectiveness of \text{MAC-SORT} tracker, we conduct ablation studies on the DanceTrack and MOT20 datasets for the MOT task. Our dataset, code, and models will be publicly available at: https://fsoft-aic.github.io/TP-GMOT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。