轻量级追踪框架通过密集时序采样提升目标定位精度
Exploring Reliable Spatiotemporal Dependencies for Efficient Visual Tracking
- 采用稠密视频采样与时间传播的时空标记,增强帧间信息利用
- 在GOT-10k上达到192帧/秒(GPU)实时性能,媲美非实时强模型
- 适合追求高精度与实时性的视觉追踪应用场景
基于Transformer的轻量级目标追踪方法在多个基准上取得了新标准,其依赖注意力机制的全局感受野和强大特征提取能力。然而,现有方法在训练中普遍采用稀疏采样——每序列仅使用一个模板图像和一个搜索图像——无法充分挖掘视频中的时空信息,限制了性能并造成轻量级与高性能追踪器之间的差距。为缩小这一差距并保持实时效率,本文提出STDTrack框架,首次将可靠的时空依赖融入轻量级追踪器。该方法实现稠密视频采样以最大化时空信息利用,引入时间传播的时空标记引导逐帧特征提取,并设计多帧信息融合模块(MFIFM),结合历史上下文增强目标状态表征。MFIFM作用于构建的时空标记维持器(STM),通过基于质量的更新机制确保信息可靠性。针对目标尺度变化问题,设计多尺度预测头以动态适应不同大小物体。大量实验表明,STDTrack在六个基准上均达领先水平。尤其在GOT-10k上,其性能可媲美某些非实时高性能追踪器(如MixFormer),同时保持192 FPS(GPU)和41 FPS(CPU)的实时速度。
原文摘要 · Abstract (English)
Recent advances in transformer-based lightweight object tracking have established new standards across benchmarks, leveraging the global receptive field and powerful feature extraction capabilities of attention mechanisms. Despite these achievements, existing methods universally employ sparse sampling during training--utilizing only one template and one search image per sequence--which fails to comprehensively explore spatiotemporal information in videos. This limitation constrains performance and cause the gap between lightweight and high-performance trackers. To bridge this divide while maintaining real-time efficiency, we propose STDTrack, a framework that pioneers the integration of reliable spatiotemporal dependencies into lightweight trackers. Our approach implements dense video sampling to maximize spatiotemporal information utilization. We introduce a temporally propagating spatiotemporal token to guide per-frame feature extraction. To ensure comprehensive target state representation, we disign the Multi-frame Information Fusion Module (MFIFM), which augments current dependencies using historical context. The MFIFM operates on features stored in our constructed Spatiotemporal Token Maintainer (STM), where a quality-based update mechanism ensures information reliability. Considering the scale variation among tracking targets, we develop a multi-scale prediction head to dynamically adapt to objects of different sizes. Extensive experiments demonstrate state-of-the-art results across six benchmarks. Notably, on GOT-10k, STDTrack rivals certain high-performance non-real-time trackers (e.g., MixFormer) while operating at 192 FPS(GPU) and 41 FPS(CPU).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。