首个端到端开放词汇多目标追踪模型，同时建模运动、外观与类别信息。
OVTR: End-to-End Open-Vocabulary Multiple Object Tracking with Transformer
- 设计类别信息传播机制，跨帧保持类别一致性
- 在开放词汇MOT基准上超越现有方法，推理更快且预处理更少
- 适合需要跨场景泛化能力的追踪应用
开放词汇多目标追踪旨在使追踪器在训练中泛化到未见类别，从而适用于多种真实场景。然而，现有方法受限于框架结构、帧间感知孤立及模态交互不足，影响分类与追踪性能。本文提出OVTR（基于Transformer的端到端开放词汇多目标追踪），首次实现运动、外观与类别信息的联合建模。为保证分类稳定与持续追踪，设计类别信息传播（CIP）策略，在后续帧中建立多级高级类别先验。引入双分支结构增强泛化能力，深化跨模态交互，并在解码器中加入保护机制提升性能。实验表明，该方法在开放词汇MOT基准上超越现有追踪器，推理速度更快，预处理需求显著降低。模型迁移至另一数据集也表现出强适应性。代码与模型已开源。
原文摘要 · Abstract (English)
Open-vocabulary multiple object tracking aims to generalize trackers to unseen categories during training, enabling their application across a variety of real-world scenarios. However, the existing open-vocabulary tracker is constrained by its framework structure, isolated frame-level perception, and insufficient modal interactions, which hinder its performance in open-vocabulary classification and tracking. In this paper, we propose OVTR (End-to-End Open-Vocabulary Multiple Object Tracking with TRansformer), the first end-to-end open-vocabulary tracker that models motion, appearance, and category simultaneously. To achieve stable classification and continuous tracking, we design the CIP (Category Information Propagation) strategy, which establishes multiple high-level category information priors for subsequent frames. Additionally, we introduce a dual-branch structure for generalization capability and deep multimodal interaction, and incorporate protective strategies in the decoder to enhance performance. Experimental results show that our method surpasses previous trackers on the open-vocabulary MOT benchmark while also achieving faster inference speeds and significantly reducing preprocessing requirements. Moreover, the experiment transferring the model to another dataset demonstrates its strong adaptability. Models and code are released at https://github.com/jinyanglii/OVTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。