仅用2D框训练,实现单目3D目标检测,大幅降低标注成本。
Weak Cube R-CNN: Weakly Supervised 3D Detection using only 2D Bounding Boxes
- 利用2D框与3D立方体投影关系,通过预训练模型估计深度和朝向。
- 在SUN RGB-D上性能优于基线,使用相同标注时间但无需3D标签。
- 适合缺乏3D标注数据的场景,为低成本3D检测提供新思路。
单目3D目标检测在机器人和虚拟现实等领域至关重要,但传统方法依赖昂贵且耗时的3D标注数据。本文提出Weak Cube R-CNN,仅需2D边界框标注即可训练,通过利用2D投影与3D立方体的关系实现3D预测。方法借助预训练的冻结2D基础模型估计训练集中的深度与方向信息,并将其作为伪真值用于训练。设计的损失函数融合外部模型输出,避免直接使用3D标签,隐式传递知识。在SUN RGB-D数据集上的实验表明,该方法在等量标注时间下性能优于基准模型。虽不适用于厘米级精度要求,但为后续研究提供了坚实基础。
原文摘要 · Abstract (English)
Monocular 3D object detection is an essential task in computer vision, and it has several applications in robotics and virtual reality. However, 3D object detectors are typically trained in a fully supervised way, relying extensively on 3D labeled data, which is labor-intensive and costly to annotate. This work focuses on weakly-supervised 3D detection to reduce data needs using a monocular method that leverages a singlecamera system over expensive LiDAR sensors or multi-camera setups. We propose a general model Weak Cube R-CNN, which can predict objects in 3D at inference time, requiring only 2D box annotations for training by exploiting the relationship between 2D projections of 3D cubes. Our proposed method utilizes pre-trained frozen foundation 2D models to estimate depth and orientation information on a training set. We use these estimated values as pseudo-ground truths during training. We design loss functions that avoid 3D labels by incorporating information from the external models into the loss. In this way, we aim to implicitly transfer knowledge from these large foundation 2D models without having access to 3D bounding box annotations. Experimental results on the SUN RGB-D dataset show increased performance in accuracy compared to an annotation time equalized Cube R-CNN baseline. While not precise for centimetre-level measurements, this method provides a strong foundation for further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。