用极少标注实现高精度单目3D目标追踪，突破标注成本瓶颈。
Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision
- 分两步：先匹配2D图像中的目标，再估计3D几何结构。
- 仅用每条轨迹4个标注，追踪准确率提升15.5个百分点。
- 适合标注稀缺但需高精度3D感知的自动驾驶场景。
单目3D目标追踪旨在跨视频帧估计一致的3D目标姿态，使自主系统能理解场景动态。然而现有先进方法依赖长视频序列的密集3D标注，获取成本高且难以扩展。本文提出首个稀疏监督的单目3D追踪框架，将任务分解为2D查询匹配与3D几何估计两个阶段。二者利用图像序列的时空一致性，增强少量标注样本，学习丰富的2D和3D场景表征。基于这些表征，模型可自动生成高质量3D伪标签覆盖全视频，将稀疏监督转化为稠密跟踪标注。这使得现有全监督追踪器可在极低标注密度下有效运行。在KITTI和nuScenes数据集上的大量实验表明，本方法显著提升追踪性能，最多仅需每轨迹4个真实标注，准确率提升达15.50个百分点。
原文摘要 · Abstract (English)
Monocular 3D object tracking aims to estimate temporally consistent 3D object poses across video frames, enabling autonomous agents to reason about scene dynamics. However, existing state-of-the-art approaches are fully supervised and rely on dense 3D annotations over long video sequences, which are expensive to obtain and difficult to scale. In this work, we address this fundamental limitation by proposing the first sparsely supervised framework for monocular 3D object tracking. Our approach decomposes the task into two sequential sub-problems: 2D query matching and 3D geometry estimation. Both components leverage the spatio-temporal consistency of image sequences to augment a sparse set of labeled samples and learn rich 2D and 3D representations of the scene. Leveraging these learned cues, our model automatically generates high-quality 3D pseudolabels across entire videos, effectively transforming sparse supervision into dense 3D track annotations. This enables existing fully-supervised trackers to effectively operate under extreme label sparsity. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method significantly improves tracking performance, achieving an improvement of up to 15.50 p.p. while using at most four ground truth annotations per track.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。