用统一的3D高斯表示实现自动驾驶视觉预训练，兼顾几何与纹理。
GaussianPretrain: A Simple Unified 3D Gaussian Representation for Visual Pre-training in Autonomous Driving
- 将3D高斯锚点视为体素激光点，统一建模场景几何与纹理。
- 比NeRF方法快40.6%，仅需70%显存，3D检测NDS提升7.05%。
- 适合需要高效、全面场景理解的自动驾驶视觉预训练任务。
自监督学习在图像处理中已取得显著进展，而自动驾驶领域的视觉预训练仍处于起步阶段。现有方法通常聚焦于学习几何场景信息，忽视纹理或分开处理二者，阻碍了对场景的全面理解。为此，我们提出GaussianPretrain，一种新颖的预训练范式，通过统一整合几何与纹理表示，实现对场景的全方位理解。将3D高斯锚点概念化为体素激光点，该方法深化了对场景的空间结构与纹理细节的理解，显著提升预训练性能：相比基于NeRF的方法UniPAD，速度提升40.6%，显存占用仅为70%。我们在多个3D感知任务中验证了其有效性，表现显著提升——3D目标检测的NDS提高7.05%，高清地图构建的mAP提升1.9%，占据预测准确率提升0.8%。这些成果凸显GaussianPretrain在理论创新与实际应用中的潜力，推动自动驾驶视觉预训练发展。源代码将在https://github.com/Public-BOTs/GaussianPretrain公开。
原文摘要 · Abstract (English)
Self-supervised learning has made substantial strides in image processing, while visual pre-training for autonomous driving is still in its infancy. Existing methods often focus on learning geometric scene information while neglecting texture or treating both aspects separately, hindering comprehensive scene understanding. In this context, we are excited to introduce GaussianPretrain, a novel pre-training paradigm that achieves a holistic understanding of the scene by uniformly integrating geometric and texture representations. Conceptualizing 3D Gaussian anchors as volumetric LiDAR points, our method learns a deepened understanding of scenes to enhance pre-training performance with detailed spatial structure and texture, achieving that 40.6% faster than NeRF-based method UniPAD with 70% GPU memory only. We demonstrate the effectiveness of GaussianPretrain across multiple 3D perception tasks, showing significant performance improvements, such as a 7.05% increase in NDS for 3D object detection, boosts mAP by 1.9% in HD map construction and 0.8% improvement on Occupancy prediction. These significant gains highlight GaussianPretrain's theoretical innovation and strong practical potential, promoting visual pre-training development for autonomous driving. Source code will be available at https://github.com/Public-BOTs/GaussianPretrain
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。