无需标注框,SSTrack++通过自监督学习实现高性能视觉追踪。
Exploring Decoupled Spatio-Temporal Consistency Learning and Self-Prompting Evolution for Self-Supervised Tracking
- 设计弱到强自监督框架,缩小有标签与无标签帧的特征差距。
- 在GOT10K等数据集上提升超25.8%,逼近全监督模型性能。
- 适合无标注数据场景,适用于复杂真实追踪任务。
视觉追踪的成功主要依赖于人工标注边框的数据集,但此类标注耗费大量人力,限制了数据集规模与多样性。本文提出新型自监督追踪模型SSTrack++,旨在消除对标注框的需求。设计了一种有效的弱到强自监督训练框架,以缩小有标签与无标签帧间的特征分布差异。在此基础上,引入解耦时空一致性训练策略,通过全局空间定位与局部时序关联捕捉丰富目标信息。同时,设计自提示演化模块,在特征与决策层面联合挖掘目标外观演化规律,增强对复杂多变无标注场景的适应能力。此外,提出实例对比损失,从多视角学习实例级对应关系,无需额外标注即可提供鲁棒的实例级监督。该范式使SSTrack++不仅能以低标注成本学习通用追踪表示,还能模拟真实场景中目标的外观与运动变化。在十个基准数据集上的实验表明,SSTrack++显著超越现有自监督追踪方法,在GOT10K、LaSOT、TrackingNet上的AUC(AO)分数分别提升25.8%、21.4%和15.8%,大幅缩小与全监督追踪器的性能差距。代码已开源:https://github.com/GXNU-ZhongLab/SSTrack。
原文摘要 · Abstract (English)
The success of visual tracking has been largely driven by datasets with manual box annotations. However, these box annotations require tremendous human effort, limiting the scale and diversity of existing tracking datasets. In this work, we present a novel high-performance Self-Supervised Tracking model named SSTrack++, designed to eliminate the need of box annotations. Specifically, we design an effective weak-to-strong self-supervised training framework that aims to narrow the feature distribution gap between labeled and unlabeled frames. Building upon this framework, a decoupled spatio-temporal consistency training strategy is introduced to capture rich target information across timestamps by leveraging global spatial localization and local temporal association. Then, a self-prompting evolution module is designed to jointly mine the target appearance evolution at both the feature and decision levels, enabling more robust adaptation to complex and diverse unlabeled tracking scenarios. Furthermore, an instance contrastive loss is formulated to learn instance-level correspondences from a multi-view perspective, providing robust instance-level supervision without any additional annotations. This new design paradigm enables SSTrack++ to not only learn a generalizable tracking representation in a low-annotation self-supervised manner, but also makes it possible to simulate realistic appearance and motion variations of target instances in real-world scenarios. Extensive experiments on ten benchmark datasets demonstrate that SSTrack++ surpasses SOTA self-supervised tracking methods, achieving an improvement of more than 25.8%, 21.4%, and 15.8% in AUC (AO) score on the GOT10K, LaSOT, TrackingNet datasets, respectively, thereby significantly narrowing the performance gap with fully supervised trackers. Code is available at https://github.com/GXNU-ZhongLab/SSTrack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。