arXiv:2509.19115cs.CV2025-09TPAMI被引 6

用记忆机制实现高效在线点追踪,抗遮挡漂移表现优

Track-On2: Enhancing Online Point Tracking with Memory

  • 基于因果注意力与内存机制,逐帧处理不依赖未来帧
  • 在五个真实与合成数据集上达到当前最优,超越部分离线方法
  • 纯合成数据训练即可实现长时鲁棒性,适合实时应用

本文研究长期点追踪问题,要求在显著外观变化、运动和遮挡下跨视频帧保持点的一致识别。针对在线设置(逐帧追踪),提出Track-On2——一个简单高效的基于Transformer的模型。通过架构优化、更有效的内存利用和改进的合成训练策略,提升性能与效率。不同于依赖全序列访问或迭代更新的方法,该模型采用因果处理,通过记忆机制维持时间连贯性,关键在于无需未来帧即可应对漂移与遮挡。推理时先进行粗粒度块级分类,再细化。系统研究了合成训练设置对内存行为的影响,揭示其如何塑造长序列的时间鲁棒性。在五个合成与真实世界基准上全面验证,Track-On2取得当前最优结果,超越以往在线追踪器甚至部分利用双向上下文的强离线方法。结果表明,仅用合成数据训练的因果记忆架构是可扩展的真实点追踪方案。

原文摘要 · Abstract (English)

In this paper, we consider the problem of long-term point tracking, which requires consistent identification of points across video frames under significant appearance changes, motion, and occlusion. We target the online setting, i.e. tracking points frame-by-frame, making it suitable for real-time and streaming applications. We extend our prior model Track-On into Track-On2, a simple and efficient transformer-based model for online long-term tracking. Track-On2 improves both performance and efficiency through architectural refinements, more effective use of memory, and improved synthetic training strategies. Unlike prior approaches that rely on full-sequence access or iterative updates, our model processes frames causally and maintains temporal coherence via a memory mechanism, which is key to handling drift and occlusions without requiring future frames. At inference, we perform coarse patch-level classification followed by refinement. Beyond architecture, we systematically study synthetic training setups and their impact on memory behavior, showing how they shape temporal robustness over long sequences. Through comprehensive experiments, Track-On2 achieves state-of-the-art results across five synthetic and real-world benchmarks, surpassing prior online trackers and even strong offline methods that exploit bidirectional context. These results highlight the effectiveness of causal, memory-based architectures trained purely on synthetic data as scalable solutions for real-world point tracking. Project page: https://kuis-ai.github.io/track_on2

点追踪在线追踪记忆机制合成训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。