用对比随机游走实现自监督任意点跟踪，精度高且无需标注。
Self-Supervised Any-Point Tracking by Contrastive Random Walks
- 基于注意力的全局匹配构建时空图上的随机游走转移矩阵。
- 在TapVid上超越自监督方法DIFT，接近有监督方法性能。
- 通过循环一致性训练，避免复杂层级匹配，适合视频跟踪任务。
我们提出一种简单、自监督的任意点跟踪（TAP）方法。通过对比随机游走，训练一个全局匹配变换器，在视频中寻找循环一致的轨迹。利用变换器基于注意力的全局匹配来定义时空图上的随机游走转移矩阵。模型能够进行“所有点对”比较，从而获得高空间精度和强对比学习信号，同时避免了近期方法中复杂的分层匹配流程。为此，我们提出若干设计决策，使全局匹配架构可通过循环一致性实现自监督训练。例如，发现基于变换器的方法易受捷径解影响，并提出数据增强方案应对。该方法在TapVid基准上表现优异，显著优于先前自监督跟踪方法（如DIFT），并可与多个有监督方法相媲美。
原文摘要 · Abstract (English)
We present a simple, self-supervised approach to the Tracking Any Point (TAP) problem. We train a global matching transformer to find cycle consistent tracks through video via contrastive random walks, using the transformer's attention-based global matching to define the transition matrices for a random walk on a space-time graph. The ability to perform "all pairs" comparisons between points allows the model to obtain high spatial precision and to obtain a strong contrastive learning signal, while avoiding many of the complexities of recent approaches (such as coarse-to-fine matching). To do this, we propose a number of design decisions that allow global matching architectures to be trained through self-supervision using cycle consistency. For example, we identify that transformer-based methods are sensitive to shortcut solutions, and propose a data augmentation scheme to address them. Our method achieves strong performance on the TapVid benchmarks, outperforming previous self-supervised tracking methods, such as DIFT, and is competitive with several supervised methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。