arXiv:2605.07064cs.CV2026-05被引 1

无需框标注,用自然语言描述实现视频目标追踪。

Learning to Track Instance from Single Nature Language Description

论文配图:Learning to Track Instance from Single Nature Language Description
图 1 · 摘自论文原文
  • 动态令牌聚合模块,不均等融合视觉与语言信息。
  • 在多个基准上超越现有自监督追踪方法性能。
  • 适合无标注数据场景下的语言引导追踪研究者。

如何在不依赖任何边界框真值的情况下,仅通过自然语言描述实现视频序列中的视觉-语言(VL)目标追踪?本文提出一种自监督视觉-语言追踪方法,旨在通过自然语言描述评估追踪能力。我们引入新模型\tracker,能够根据语言描述追踪任意目标。不同于传统方法对所有语言和视觉令牌一视同仁的融合方式,我们设计了高效的动态令牌聚合模块,对每个视觉令牌进行差异化处理:首先以锚定令牌为基础,从模板帧中筛选出多个关键目标令牌;其次,依据注意力得分将选定的目标令牌合并并聚合到语言令牌中,从而消除冗余视觉噪声,增强语义对齐;最后,融合后的语言令牌作为引导信号,在搜索帧中提取潜在目标令牌,并传递至后续帧,提升时序提示效果,促使追踪器自主学习实例追踪能力。该建模方法实现了无需大规模边界框标注的语言引导追踪表示的高效自监督学习。在多个视觉-语言追踪基准上的大量实验表明,\tracker显著优于当前最先进的自监督方法。

原文摘要 · Abstract (English)

How to achieve vision-language (VL) tracking using natural language descriptions from a video sequence \textbf{without relying on any bounding-box ground truth}? In this work, we achieve this goal by tackling \textit{self-supervised VL tracking}, which aims to evaluate tracking capabilities guided by natural language descriptions. We introduce \textbf{\tracker}, a novel self-supervised VL tracker that is capable of tracking any referred object by a language description. Unlike traditional methods that equally fuse all language and visual tokens, we propose an efficient Dynamic Token Aggregation Module, which treats each visual token \textbf{unequally}. The module consists of three main steps: i) Based on an anchor token, it selects multiple important target tokens from the template frame. ii) The selected target tokens are merged according to their attention scores and aggregated into the language tokens, thereby eliminating redundant visual token noise and enhancing semantic alignment. iii) Finally, the fused language tokens serve as guiding signals to extract potential target tokens from the search frame and propagate them to subsequent frames, enhancing temporal prompts and encouraging the tracker to autonomously learn instance tracking from unlabeled videos. This new modeling approach enables the effective self-supervised learning of language-guided tracking representations without the need for large-scale bounding box annotations. Extensive experiments on VL tracking benchmarks show that {\tracker} surpasses SOTA self-supervised methods.

视觉语言追踪自监督学习自然语言控制无标注训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。