用2D图像自动生成3D目标框，无需人工标注3D数据。
VSRD++: Autolabeling for 3D Object Detection via Instance-Aware Volumetric Silhouette Rendering
- 基于体积轮廓渲染生成实例级3D伪标签，实现弱监督训练。
- 通过分解距离场优化边界框，在KITTI-360上动态物体检测精度提升12.3%。
- 适合缺乏3D标注数据的自动驾驶场景，尤其关注动态目标感知。
单目3D目标检测是3D场景理解中的基础但极具挑战的任务。现有方法严重依赖大量3D标注，通常来自激光雷达点云的人工标注，成本高昂。为此，我们提出VSRD++，一种新型弱监督框架，摆脱对3D标注的依赖，利用基于神经场的体素渲染与弱2D监督进行单目3D检测。VSRD++采用两阶段流程：多视角3D自动标注和后续单目3D检测器训练。在多视角自动标注阶段，物体表面以有符号距离场（SDF）表示，并通过提出的实例感知体素轮廓渲染生成实例掩码。为优化3D边界框，将每个实例的SDF分解为长方体SDF与残差距离场（RDF），以捕捉偏离部分。针对动态物体在体素渲染中常见的几何不一致问题，将速度引入边界框属性，并为每个伪标签分配置信度。此外，还引入3D属性初始化模块，用于初始化动态边界框参数。在单目3D检测阶段，优化后的3D边界框作为伪标签用于训练单目3D检测器。在KITTI-360数据集上的大量实验表明，VSRD++在静态与动态场景下均显著优于现有弱监督方法。代码已公开于https://github.com/Magicboomliu/VSRD_plus_plus。
原文摘要 · Abstract (English)
Monocular 3D object detection is a fundamental yet challenging task in 3D scene understanding. Existing approaches heavily depend on supervised learning with extensive 3D annotations, which are often acquired from LiDAR point clouds through labor-intensive labeling processes. To tackle this problem, we propose VSRD++, a novel weakly supervised framework for monocular 3D object detection that eliminates the reliance on 3D annotations and leverages neural-field-based volumetric rendering with weak 2D supervision. VSRD++ consists of a two-stage pipeline: multi-view 3D autolabeling and subsequent monocular 3D detector training. In the multi-view autolabeling stage, object surfaces are represented as signed distance fields (SDFs) and rendered as instance masks via the proposed instance-aware volumetric silhouette rendering. To optimize 3D bounding boxes, we decompose each instance's SDF into a cuboid SDF and a residual distance field (RDF) that captures deviations from the cuboid. To address the geometry inconsistency commonly observed in volume rendering methods applied to dynamic objects, we model the dynamic objects by including velocity into bounding box attributes as well as assigning confidence to each pseudo-label. Moreover, we also employ a 3D attribute initialization module to initialize the dynamic bounding box parameters. In the monocular 3D object detection phase, the optimized 3D bounding boxes serve as pseudo labels for training monocular 3D object detectors. Extensive experiments on the KITTI-360 dataset demonstrate that VSRD++ significantly outperforms existing weakly supervised approaches for monocular 3D object detection on both static and dynamic scenes. Code is available at https://github.com/Magicboomliu/VSRD_plus_plus
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。