arXiv:2608.14428cs.CV2026-08中稿 · ECCV被引 1

通过幻化被遮挡的激光点云结构,提升自动驾驶3D检测的鲁棒性。

GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure

论文配图:GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure
图 1 · 摘自论文原文
  • 用实例体素膨胀生成遮挡区域的潜在特征,实现局部结构幻化。
  • 在nuScenes和Waymo上显著提升稀疏扫描与少样本条件下的3D检测精度。
  • 适合追求高鲁棒性的自动驾驶感知系统研发人员使用。

从激光雷达点云进行3D目标检测是自动驾驶的核心问题。近年来自监督学习(SSL)在可扩展预训练方面取得进展,对逐点任务如语义和全景分割迁移效果良好,但对3D检测的迁移能力仍较弱。我们分析发现,现有SSL方法仅在可见表面的测量点上定义目标,未约束遮挡和未观测区域,存在可见表面偏差。该偏差足以支持逐点预测,但3D检测需对缺失结构具备鲁棒性。为此,我们提出GhostPoint,一种通过新型实例体素膨胀生成邻域潜在特征的自监督框架。在GhostPoint中,编码器处理可观测返回,额外预测器从上下文推断邻域表示。除标准编码器监督外,引入预测器层级监督:采样生成邻域中的观测(可见/掩码)体素匹配教师编码器目标,未观测体素匹配教师预测器幻化结果。该设计促使学习表征显式建模超出可观测返回的结构。在nuScenes和Waymo上的大量实验表明,该方法达到当前最优性能,尤其在稀疏扫描和有限标注条件下持续提升下游3D检测表现。

原文摘要 · Abstract (English)

3D object detection from LiDAR point clouds is a core problem in autonomous driving. Recent advances in self-supervised learning (SSL) enable scalable pretraining and transfers well to per-point tasks such as semantic and panoptic segmentation, but transfer to 3D detection remains weaker. We analyze recent SSL methods and find that most objectives are defined only on measured LiDAR returns from visible surfaces, leaving occluded and unobserved regions unconstrained. This visible-surface bias can be sufficient for point-wise prediction, but 3D detection requires robustness to missing structure. To address this gap, we propose GhostPoint, an SSL framework that hallucinates latent features in local neighborhoods around discovered instances, generated via a novel instance voxel dilation. In GhostPoint, an encoder processes observed returns, and an additional predictor infers neighborhood representations from observed context. In addition to standard encoder-level supervision, we introduce a predictor-level supervision scheme on sampled voxels from generated neighborhoods. Specifically, observed (visible/masked) voxels match teacher-encoder targets, while unobserved voxels match teacher-predictor hallucinations. This design encourages the learned representation to explicitly model structure beyond observed returns. Extensive evaluations on nuScenes and Waymo demonstrate that our method achieves state-of-the-art performance, consistently improving downstream 3D detection, especially under sparse scans and limited labels.

3D检测自监督学习激光雷达结构幻化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。