arXiv:2512.08358cs.CV2025-12NeurIPS被引 5

提出世界坐标系下的密集单目3D跟踪方法,能同时分离相机运动与物体运动。

TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels

  • 用追踪上采样器将稀疏2D轨迹扩展为稠密2D轨迹。
  • 在真实和合成数据集上实现世界坐标系下高精度稠密3D轨迹重建。
  • 适用于新出现的动态物体,适合需要完整场景理解的任务。

单目3D跟踪旨在从单目视频中长期捕捉像素在三维空间中的运动,近年来进展迅速。然而,现有方法仍难以区分相机运动与前景动态运动,且无法稠密追踪视频中新出现的动态物体。为此,我们提出TrackingWorld,一种在世界坐标系中稠密追踪几乎所有像素的新流程。首先,引入追踪上采样器,高效地将任意稀疏2D轨迹提升为稠密2D轨迹;其次,为泛化至新出现物体,对所有帧应用上采样器,并通过消除重叠区域轨迹减少冗余;最后,提出基于优化的框架,通过估计相机位姿和2D轨迹的3D坐标,将稠密2D轨迹回投影至世界坐标系下的3D轨迹。在合成与真实世界数据集上的大量评估表明,该系统在世界坐标系下实现了准确且稠密的3D跟踪。

原文摘要 · Abstract (English)

Monocular 3D tracking aims to capture the long-term motion of pixels in 3D space from a single monocular video and has witnessed rapid progress in recent years. However, we argue that the existing monocular 3D tracking methods still fall short in separating the camera motion from foreground dynamic motion and cannot densely track newly emerging dynamic subjects in the videos. To address these two limitations, we propose TrackingWorld, a novel pipeline for dense 3D tracking of almost all pixels within a world-centric 3D coordinate system. First, we introduce a tracking upsampler that efficiently lifts the arbitrary sparse 2D tracks into dense 2D tracks. Then, to generalize the current tracking methods to newly emerging objects, we apply the upsampler to all frames and reduce the redundancy of 2D tracks by eliminating the tracks in overlapped regions. Finally, we present an efficient optimization-based framework to back-project dense 2D tracks into world-centric 3D trajectories by estimating the camera poses and the 3D coordinates of these 2D tracks. Extensive evaluations on both synthetic and real-world datasets demonstrate that our system achieves accurate and dense 3D tracking in a world-centric coordinate frame.

3D跟踪单目视觉世界坐标稠密追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。