arXiv:2409.11235cs.CV2024-09ECCV被引 12

提出SLAck框架,统一建模语义、位置与外观,提升开放词汇目标追踪性能。

SLAck: Semantic, Location, and Appearance Aware Open-Vocabulary Tracking

论文配图:SLAck: Semantic, Location, and Appearance Aware Open-Vocabulary Tracking
图 1 · 摘自论文原文
  • 早期联合建模语义、位置与外观信息,构建轻量时空图融合多源线索。
  • 在Open-Vocabulary MOT和TAO TETA上超越现有最佳方法,尤其对新类别表现优异。
  • 无需复杂后处理,适合大规模开放词汇追踪场景,如自动驾驶与视频理解。

开放词汇多目标追踪(Open-vocabulary MOT)旨在使追踪器泛化到训练集外的新类别。当前最优方法主要依赖纯外观匹配,但在大规模词汇场景下,由于运动模式复杂且新类物体分类不稳定,现有方法往往忽略运动与语义线索,或仅在最终匹配阶段通过启发式方式融合。本文提出统一框架SLAck,将语义、位置与外观先验在关联的早期步骤中联合考虑,并通过轻量级时空对象图学习如何整合所有有效信息。该方法消除了复杂的后期融合启发式规则,显著提升了大规模开放词汇追踪的关联性能。不依赖额外技巧,我们在Open-vocabulary MOT和TAO TETA基准上均超越先前最先进方法,在新类别追踪上表现突出。代码已开源。

原文摘要 · Abstract (English)

Open-vocabulary Multiple Object Tracking (MOT) aims to generalize trackers to novel categories not in the training set. Currently, the best-performing methods are mainly based on pure appearance matching. Due to the complexity of motion patterns in the large-vocabulary scenarios and unstable classification of the novel objects, the motion and semantics cues are either ignored or applied based on heuristics in the final matching steps by existing methods. In this paper, we present a unified framework SLAck that jointly considers semantics, location, and appearance priors in the early steps of association and learns how to integrate all valuable information through a lightweight spatial and temporal object graph. Our method eliminates complex post-processing heuristics for fusing different cues and boosts the association performance significantly for large-scale open-vocabulary tracking. Without bells and whistles, we outperform previous state-of-the-art methods for novel classes tracking on the open-vocabulary MOT and TAO TETA benchmarks. Our code is available at \href{https://github.com/siyuanliii/SLAck}{github.com/siyuanliii/SLAck}.

目标追踪开放词汇多目标追踪时空图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。