让视觉语言追踪更鲁棒,动态对齐目标与上下文线索。
ATCTrack: Aligning Target-Context Cues with Dynamic Target States for Robust Vision-Language Tracking
- 建模目标-上下文关系,动态更新视觉线索。
- 精准识别目标词,自适应融合上下文词。
- 在主流数据集上达到新最优性能,适合复杂长时追踪场景。
视觉语言追踪旨在通过初始帧中的模板块和语言描述,在视频序列中定位目标物体。为实现鲁棒追踪,尤其在反映真实世界条件的复杂长期场景(如最近提出的MGIT)中,不仅需刻画目标特征,还需利用相关上下文特征。然而,初始提示生成的视觉与文本目标-上下文线索仅与初始目标状态对齐,而目标状态随时间动态变化,尤其在复杂长期序列中更为显著,导致这些线索难以持续引导视觉语言追踪器(VLTs)。此外,对于表达多样的文本提示,实验表明现有VLTs难以区分哪些词语属于目标或上下文,加剧了文本线索的使用难度。本文提出新型追踪器ATCTrack,通过全面的目标-上下文特征建模,实现多模态线索与动态目标状态的对齐,从而实现鲁棒追踪。具体而言:(1) 视觉模态上,提出有效的时间视觉目标-上下文建模方法,提供及时视觉线索;(2) 文本模态上,仅基于文本内容实现目标词精准识别,并设计创新的上下文词校准方法,自适应利用辅助上下文词;(3) 在主流基准上开展大量实验,ATCTrack取得新的最先进性能。代码与模型将发布于:https://github.com/XiaokunFeng/ATCTrack。
原文摘要 · Abstract (English)
Vision-language tracking aims to locate the target object in the video sequence using a template patch and a language description provided in the initial frame. To achieve robust tracking, especially in complex long-term scenarios that reflect real-world conditions as recently highlighted by MGIT, it is essential not only to characterize the target features but also to utilize the context features related to the target. However, the visual and textual target-context cues derived from the initial prompts generally align only with the initial target state. Due to their dynamic nature, target states are constantly changing, particularly in complex long-term sequences. It is intractable for these cues to continuously guide Vision-Language Trackers (VLTs). Furthermore, for the text prompts with diverse expressions, our experiments reveal that existing VLTs struggle to discern which words pertain to the target or the context, complicating the utilization of textual cues. In this work, we present a novel tracker named ATCTrack, which can obtain multimodal cues Aligned with the dynamic target states through comprehensive Target-Context feature modeling, thereby achieving robust tracking. Specifically, (1) for the visual modality, we propose an effective temporal visual target-context modeling approach that provides the tracker with timely visual cues. (2) For the textual modality, we achieve precise target words identification solely based on textual content, and design an innovative context words calibration method to adaptively utilize auxiliary context words. (3) We conduct extensive experiments on mainstream benchmarks and ATCTrack achieves a new SOTA performance. The code and models will be released at: https://github.com/XiaokunFeng/ATCTrack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。