arXiv:2503.06471cs.CV2025-03ICCV被引 3

轻量级模型实现视频中密集点的实时追踪,兼顾精度与速度。

Online Dense Point Tracking with Streaming Memory

  • 引入流式记忆机制,融合短期运动动态与可见性引导传播。
  • 在CVO基准上达到顶尖精度,参数量减少10倍,速度提升2倍以上。
  • 适合需要低延迟、高效率的在线视频处理场景。

密集点追踪需在视频中持续追踪初始帧的每个点,即使存在遮挡也需保持稳定。传统方法依赖光流估计长程运动,易出现外观漂移且缺乏时间一致性。近期算法多采用滑动窗口间接传递信息,效率低下且难以支撑长程追踪。为此,本文提出轻量快速的SPOT框架,包含定制化记忆读取模块、短时运动动态感知的记忆单元及可见性引导的点云投射模块。该设计使SPOT在CVO基准上实现当前最优精度,并在TAP-Vid和RoboTAP等稀疏追踪基准上表现媲美甚至超越离线模型。值得注意的是,仅具原模型1/10参数量的SPOT运行速度至少快2倍,同时在CVO上保持最佳性能。代码与模型将公开于:https://dqiaole.github.io/SPOT/

原文摘要 · Abstract (English)

Dense point tracking is a challenging task requiring the continuous tracking of every point in the initial frame throughout a substantial portion of a video, even in the presence of occlusions. Traditional methods use optical flow models to directly estimate long-range motion, but they often suffer from appearance drifting without considering temporal consistency. Recent point tracking algorithms usually depend on sliding windows for indirect information propagation from the first frame to the current one, which is slow and less effective for long-range tracking. To account for temporal consistency and enable efficient information propagation, we present a lightweight and fast model with \textbf{S}treaming memory for dense \textbf{PO}int \textbf{T}racking and online video processing. The \textbf{SPOT} framework features three core components: a customized memory reading module for feature enhancement, a sensory memory for short-term motion dynamics modeling, and a visibility-guided splatting module for accurate information propagation. This combination enables SPOT to perform dense point tracking with state-of-the-art accuracy on the CVO benchmark, as well as comparable or superior performance to offline models on sparse tracking benchmarks such as TAP-Vid and RoboTAP. Notably, SPOT with 10$\times$ smaller parameter numbers operates at least 2$\times$ faster than previous state-of-the-art models while maintaining the best performance on CVO. We will release the models and codes at: https://dqiaole.github.io/SPOT/.

点追踪流式记忆实时处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。