用可学习的轨迹标记实现高效鲁棒跟踪,速度与精度兼得
Robust Tracking via Mamba-based Context-aware Token Learning
- 分离时序与外观建模,仅用代表性标记捕捉时间关系
- 基于Mamba的时序模块在滑动窗口内交互,实时感知目标变化
- 适合追求实时性与高鲁棒性的视觉跟踪应用场景
如何在性能与计算成本间取得平衡是跟踪器设计的关键。现有主流方法通常通过输入越来越多的图像(或特征)来融合时序与外观信息,导致模型计算开销大、学习负担重,并引入大量无用甚至干扰的信息。为此,我们提出一种简单而鲁棒的跟踪器:将时序信息学习与外观建模分离,从一组代表性标记中提取时序关系,而非多个图像(或特征)。具体而言,每帧设置一个轨迹标记用于收集目标外观信息;设计基于Mamba的时序模块,使轨迹标记在滑动窗口内与其他标记交互,该模块包含具备自回归特性的Mamba层和具有强全局感知能力的交叉注意力层,充分交互以感知目标外观变化与运动趋势;最后,轨迹标记引导头部调整外观特征以完成最终预测。实验表明,该方法在多个基准上表现优异,且达到实时速度。代码与训练模型将在https://github.com/GXNU-ZhongLab/TemTrack公开。
原文摘要 · Abstract (English)
How to make a good trade-off between performance and computational cost is crucial for a tracker. However, current famous methods typically focus on complicated and time-consuming learning that combining temporal and appearance information by input more and more images (or features). Consequently, these methods not only increase the model's computational source and learning burden but also introduce much useless and potentially interfering information. To alleviate the above issues, we propose a simple yet robust tracker that separates temporal information learning from appearance modeling and extracts temporal relations from a set of representative tokens rather than several images (or features). Specifically, we introduce one track token for each frame to collect the target's appearance information in the backbone. Then, we design a mamba-based Temporal Module for track tokens to be aware of context by interacting with other track tokens within a sliding window. This module consists of a mamba layer with autoregressive characteristic and a cross-attention layer with strong global perception ability, ensuring sufficient interaction for track tokens to perceive the appearance changes and movement trends of the target. Finally, track tokens serve as a guidance to adjust the appearance feature for the final prediction in the head. Experiments show our method is effective and achieves competitive performance on multiple benchmarks at a real-time speed. Code and trained models will be available at https://github.com/GXNU-ZhongLab/TemTrack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。