arXiv:2601.01925cs.CV2026-01被引 2

用自回归框架把多目标追踪变成序列生成,灵活适配新任务

AR-MOT: Autoregressive Multi-object Tracking

  • 将追踪任务转为大模型的序列生成,无需定制输出头
  • 在MOT17和DanceTrack上达到顶尖水平,精度接近现有最优方法
  • 支持指令驱动与多模态扩展,适合未来通用追踪系统

随着多目标追踪(MOT)任务向更通用、多模态方向发展,现有方法依赖固定输出头和专用追踪流程,难以适应多样任务与新型追踪范式。为此,我们提出AR-MOT,一种基于大语言模型(LLM)的自回归范式,将MOT建模为序列生成任务,实现无需任务特定头的结构化输出。为增强区域级视觉感知,引入基于预训练检测器的物体分词器;为缓解全局与局部特征错位,设计区域感知对齐(RAA)模块;为支持长期追踪,构建时序记忆融合(TMF)模块以缓存历史物体标记。该方法具备强可扩展性,通过修改输出格式即可集成新模态或指令,无需改动模型架构。在MOT17和DanceTrack上的大量实验验证了其可行性,性能媲美当前最先进方法,为构建更通用、灵活的追踪系统奠定基础。

原文摘要 · Abstract (English)

As multi-object tracking (MOT) tasks continue to evolve toward more general and multi-modal scenarios, the rigid and task-specific architectures of existing MOT methods increasingly hinder their applicability across diverse tasks and limit flexibility in adapting to new tracking formulations. Most approaches rely on fixed output heads and bespoke tracking pipelines, making them difficult to extend to more complex or instruction-driven tasks. To address these limitations, we propose AR-MOT, a novel autoregressive paradigm that formulates MOT as a sequence generation task within a large language model (LLM) framework. This design enables the model to output structured results through flexible sequence construction, without requiring any task-specific heads. To enhance region-level visual perception, we introduce an Object Tokenizer based on a pretrained detector. To mitigate the misalignment between global and regional features, we propose a Region-Aware Alignment (RAA) module, and to support long-term tracking, we design a Temporal Memory Fusion (TMF) module that caches historical object tokens. AR-MOT offers strong potential for extensibility, as new modalities or instructions can be integrated by simply modifying the output sequence format without altering the model architecture. Extensive experiments on MOT17 and DanceTrack validate the feasibility of our approach, achieving performance comparable to state-of-the-art methods while laying the foundation for more general and flexible MOT systems.

多目标追踪自回归大模型序列生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。