用3D伪标签提升自动驾驶环境感知,显著改善物体定位与深度估计。
Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perception
- 基于多视角融合生成3D伪标签,直接在三维空间监督模型。
- 在Occ3D-nuScenes上使mIoU提升45%,RayIoU提升49%。
- 无需复杂渲染,适合追求高效训练的自动驾驶研发团队。
在自动驾驶感知中,安全不仅关乎驾驶员,也涉及行人、骑行者等弱势道路使用者。传统鸟瞰图表示会丢失垂直信息,导致3D目标定位模糊,影响运动规划与场景预测。相比之下,语义占用可提供完整的3D环境表征。现有自监督方法依赖2D伪标签,虽提升了可扩展性,但常需新型视图合成、跨视图渲染和深度估计,带来高计算与内存开销。为此,我们提出Easy3D-Labels,利用Grounded-SAM与Metric3Dv2生成3D伪真值标签,并通过时间聚合实现稠密化,支持直接在3D空间进行监督。该方法可无缝集成至现有模型,应用于OccNeRF时,在Occ3D-nuScenes数据集上使mIoU提升45%,RayIoU提升49%。此外,我们还构建了仅依赖此类3D伪标签的轻量模型EasyOcc,避免复杂渲染,取得15.7 mIoU的性能。实验表明,该方法有效减少物体重复并提升深度估计精度,尤以RayIoU指标体现。
原文摘要 · Abstract (English)
In perception for automated vehicles, safety is critical not only for the driver but also for other agents in the scene, particularly vulnerable road users such as pedestrians and cyclists. Previous representation methods, such as Bird's Eye View, collapse vertical information, leading to ambiguity in 3D object localisation and limiting accurate understanding of the environment for downstream tasks such as motion planning and scene forecasting. In contrast, semantic occupancy provides a full 3D representation of the surroundings, addressing these limitations. Unlike supervised methods that rely on manually annotated data, self-supervised approaches use 2D pseudo-labels, improving scalability by reducing the need for labour-intensive annotation. Consequently, such models employ techniques such as novel view synthesis, cross-view rendering, and depth estimation to allow for model supervision against the 2D labels. However, such approaches often incur high computational and memory costs during training, especially for novel view synthesis. To address these issues, we propose Easy3D-Labels, which are 3D pseudo-ground-truth labels generated using Grounded-SAM and Metric3Dv2, with temporal aggregation for densification, permitting supervision directly in 3D space. Easy3D-Labels can be readily integrated into existing models to provide model supervision, yielding substantial performance gains, with mIoU increasing by 45% and RayIoU by 49% when applied to OccNeRF on the Occ3D-nuScenes dataset. Additionally, we introduce EasyOcc, a streamlined model trained solely on these 3D pseudo-labels, avoiding the need for complex rendering strategies and achieving 15.7 mIoU on Occ3D-nuScenes. Easy3D-Labels improve scene understanding by reducing object duplication and enhancing depth estimation accuracy, as reflected by improvements in the RayIoU metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。