用3D高斯点云融合多视角图像与激光雷达,提升自动驾驶定位精度
GSPR: Multimodal Place Recognition Using 3D Gaussian Splatting for Autonomous Driving
- 通过高斯点云显式融合图像与激光雷达数据
- 在三个数据集上达到最新最优性能,定位准确率显著提升
- 适合研究多模态感知与自动驾驶定位的开发者使用
位置识别是自动驾驶车辆在无卫星信号环境实现定位的关键技术。近年来,多模态位置识别方法受到关注,通过融合不同传感器的互补信息克服单模态系统的局限性。然而,多数现有方法在特征或描述子层面进行跨模态融合,缺乏可解释性。最近提出的3D高斯点云渲染为多模态融合提供了新思路,能将不同模态统一到显式的场景表示中。本文提出基于3D高斯点云的多模态位置识别网络GSPR,通过提出的多模态高斯点云渲染,将多视图RGB图像与激光雷达点云融合为时空统一的场景表示。设计包含3D图卷积与Transformer的网络,从高斯场景中提取时空特征与全局描述子用于位置识别。在三个数据集上的大量实验表明,该方法能有效利用摄像头与激光雷达的互补优势,在保持良好泛化能力的同时,实现当前最优的位置识别性能。开源代码将发布于https://github.com/QiZS-BIT/GSPR。
原文摘要 · Abstract (English)
Place recognition is a crucial component that enables autonomous vehicles to obtain localization results in GPS-denied environments. In recent years, multimodal place recognition methods have gained increasing attention. They overcome the weaknesses of unimodal sensor systems by leveraging complementary information from different modalities. However, most existing methods explore cross-modality correlations through feature-level or descriptor-level fusion, suffering from a lack of interpretability. Conversely, the recently proposed 3D Gaussian Splatting provides a new perspective on multimodal fusion by harmonizing different modalities into an explicit scene representation. In this paper, we propose a 3D Gaussian Splatting-based multimodal place recognition network dubbed GSPR. It explicitly combines multi-view RGB images and LiDAR point clouds into a spatio-temporally unified scene representation with the proposed Multimodal Gaussian Splatting. A network composed of 3D graph convolution and transformer is designed to extract spatio-temporal features and global descriptors from the Gaussian scenes for place recognition. Extensive evaluations on three datasets demonstrate that our method can effectively leverage complementary strengths of both multi-view cameras and LiDAR, achieving SOTA place recognition performance while maintaining solid generalization ability. Our open-source code will be released at https://github.com/QiZS-BIT/GSPR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。