给Transformer追踪器加了跨帧反馈,提升跟踪精度且几乎不增加参数。
FeedbackTrack: Visual-Cortex-Inspired Cross-Frame Feedback for Transformer Tracking

- 引入跨帧稀疏反馈机制,利用前一帧中间状态调节当前帧处理。
- 在LaSOT和GOT-10k上分别达83.4 AO和79.1 AUC,参数增益低于1%。
- 适合需要高精度、低开销的实时目标跟踪场景。
视觉目标跟踪需要有效的时序整合,但大多数Transformer追踪器仍依赖纯前馈特征提取。现有时序机制通常更新模板、提示、查询或预测状态,而中间表示很少被重用于调节对应处理阶段。我们提出 extbf{FeedbackTrack},一种受视觉皮层启发的框架,将稀疏、分组对齐的跨帧反馈引入预训练Transformer追踪器。前一帧的中间状态被分离、缓存,并通过两条轻量路径返回当前帧的对应Transformer组:查询反馈用于令牌级查询调制,门控反馈用于上下文相关的特征调制。FeedbackTrack仅添加固定大小的一帧缓存,保持原有追踪流程不变。在SPMTrack和ARTrackV2上,FeedbackTrack在五个主干配置下均持续提升,在LaSOT和GOT-10k上分别取得83.4 AO和79.1 AUC,参数增加少于1%。控制对比显示,跨帧反馈比同帧调制高出1.8–3.2 AO点,表明增益主要来自递归历史信息。进一步分析揭示反馈强度存在非均匀的深度依赖结构,凸显递归反馈在Transformer追踪中的有效性。
原文摘要 · Abstract (English)
Visual object tracking requires effective temporal integration, yet most Transformer trackers still rely on predominantly feed-forward feature extraction. Existing temporal mechanisms typically update templates, prompts, queries, or prediction states, while intermediate representations are rarely reused to modulate corresponding processing stages. We propose \textbf{FeedbackTrack}, a visual-cortex-inspired framework that introduces sparse, group-level layer-aligned cross-frame feedback into pretrained Transformer trackers. Previous-frame intermediate states are detached, cached, and returned to corresponding Transformer groups in the current frame through two lightweight pathways: Query Feedback for token-level query modulation and Gate Feedback for context-dependent feature modulation. FeedbackTrack preserves the original tracking pipeline with only a fixed-size one-frame cache. Across SPMTrack and ARTrackV2, FeedbackTrack consistently improves five backbone configurations on LaSOT and GOT-10k, achieving 83.4 AO and 79.1 AUC with SPMTrack-G while adding less than 1\% parameters. Controlled comparisons show that cross-frame feedback outperforms same-frame modulation by 1.8--3.2 AO points, demonstrating that the gains mainly come from recurrent historical information. Further analysis reveals a non-uniform depth-dependent organization of learned feedback strengths, highlighting the effectiveness of recurrent feedback for Transformer tracking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。