DELTA实现单目视频中像素级3D运动的高效长序列追踪。
DELTA: Dense Efficient Long-range 3D Tracking for any video
- 采用全局-局部注意力机制与Transformer上采样，实现高分辨率3D追踪。
- 速度比之前方法快8倍以上，同时保持最先进精度。
- 适合需要精细3D运动追踪的长期视频应用。
从单目视频中稠密追踪3D运动仍具挑战性，尤其在长序列中实现像素级精度时。本文提出DELTA，一种新型方法，可高效追踪视频中每个像素的3D空间位置，实现整段视频的精确运动估计。该方法首先利用联合全局-局部注意力机制进行低分辨率追踪，再通过基于Transformer的上采样器生成高分辨率预测。与现有方法相比，其计算效率更高、追踪更稠密，运行速度超过此前方法8倍，同时达到最新技术水平。此外，我们研究了深度表示对追踪性能的影响，发现对数深度（log-depth）为最优选择。大量实验表明，DELTA在多个基准测试中均取得2D和3D稠密追踪的新纪录，为需要细粒度、长时序3D运动追踪的应用提供了稳健解决方案。
原文摘要 · Abstract (English)
Tracking dense 3D motion from monocular videos remains challenging, particularly when aiming for pixel-level precision over long sequences. We introduce DELTA, a novel method that efficiently tracks every pixel in 3D space, enabling accurate motion estimation across entire videos. Our approach leverages a joint global-local attention mechanism for reduced-resolution tracking, followed by a transformer-based upsampler to achieve high-resolution predictions. Unlike existing methods, which are limited by computational inefficiency or sparse tracking, DELTA delivers dense 3D tracking at scale, running over 8x faster than previous methods while achieving state-of-the-art accuracy. Furthermore, we explore the impact of depth representation on tracking performance and identify log-depth as the optimal choice. Extensive experiments demonstrate the superiority of DELTA on multiple benchmarks, achieving new state-of-the-art results in both 2D and 3D dense tracking tasks. Our method provides a robust solution for applications requiring fine-grained, long-term motion tracking in 3D space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。