用形变替代匹配,实现高效精准的密集点跟踪
CoWTracker: Tracking by Warping instead of Correlation
- 通过特征形变迭代优化轨迹,避开耗时的特征相关计算
- 在TAP-Vid等基准上达到顶尖性能,且在光流任务中表现优异
- 适合需要高效稠密跟踪与多任务统一建模的研究者
密集点跟踪是计算机视觉中的基础问题,广泛应用于视频分析和机器人操作。现有先进追踪器通常依赖代价体积进行帧间特征匹配,但该方法的空间分辨率呈二次复杂度,限制了可扩展性和效率。本文提出一种新方法,摒弃代价体积,改用形变策略:基于当前轨迹估计,将目标帧的特征形变至查询帧。结合变压器架构实现所有轨迹的联合时空推理,无需计算特征相关性即可建立长程对应关系。模型结构简洁,在TAP-Vid-DAVIS、TAP-Vid-Kinetics和Robo-TAP等标准稠密点跟踪基准上达到最先进水平。尤为突出的是,该模型在光流任务中表现卓越,部分情况下超越专用方法,在Sintel、KITTI和Spring基准上取得领先。结果表明,基于形变的架构可统一稠密点跟踪与光流估计。
原文摘要 · Abstract (English)
Dense point tracking is a fundamental problem in computer vision, with applications ranging from video analysis to robotic manipulation. State-of-the-art trackers typically rely on cost volumes to match features across frames, but this approach incurs quadratic complexity in spatial resolution, limiting scalability and efficiency. In this paper, we propose \method, a novel dense point tracker that eschews cost volumes in favor of warping. Inspired by recent advances in optical flow, our approach iteratively refines track estimates by warping features from the target frame to the query frame based on the current estimate. Combined with a transformer architecture that performs joint spatiotemporal reasoning across all tracks, our design establishes long-range correspondences without computing feature correlations. Our model is simple and achieves state-of-the-art performance on standard dense point tracking benchmarks, including TAP-Vid-DAVIS, TAP-Vid-Kinetics, and Robo-TAP. Remarkably, the model also excels at optical flow, sometimes outperforming specialized methods on the Sintel, KITTI, and Spring benchmarks. These results suggest that warping-based architectures can unify dense point tracking and optical flow estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。