无需点云累积,单帧激光雷达与图像精准配准。
Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching
- 直接基于投影的无检测框架实现点像素匹配。
- 在nuScenes上超越依赖点云累积的方法,单帧表现更优。
- 引入可重复性评分提升稀疏输入下的匹配鲁棒性。
激光雷达点云与相机图像之间的点像素配准是自动驾驶与机器人感知中的基础但具有挑战性的任务。主要难点在于非结构化点云与结构化图像间的模态差异,尤其在稀疏单帧激光雷达条件下。现有方法通常分别提取点云与图像特征,再依赖手工或学习的匹配策略,这种分离编码难以有效弥合模态差距,且对单帧激光雷达的稀疏性和噪声敏感,常需点云累积或额外先验以提高可靠性。受无检测匹配范式进展启发,我们重新审视基于投影的方法,提出直接点像素匹配的无检测框架。为进一步提升匹配可靠性,引入可重复性评分机制,作为软可见性先验,引导网络抑制低强度变化区域的不可靠匹配,增强稀疏输入下的鲁棒性。在KITTI、nuScenes和MIAS-LCEC-TF70基准上的大量实验表明,本方法达到最先进性能,在nuScenes上甚至优于依赖点云累积的方法,仅使用单帧激光雷达即实现卓越效果。
原文摘要 · Abstract (English)
Point-pixel registration between LiDAR point clouds and camera images is a fundamental yet challenging task in autonomous driving and robotic perception. A key difficulty lies in the modality gap between unstructured point clouds and structured images, especially under sparse single-frame LiDAR settings. Existing methods typically extract features separately from point clouds and images, then rely on hand-crafted or learned matching strategies. This separate encoding fails to bridge the modality gap effectively, and more critically, these methods struggle with the sparsity and noise of single-frame LiDAR, often requiring point cloud accumulation or additional priors to improve reliability. Inspired by recent progress in detector-free matching paradigms, we revisit the projection-based approach and introduce the detector-free framework for direct point-pixel matching between LiDAR and camera views. To further enhance matching reliability, we introduce a repeatability scoring mechanism that acts as a soft visibility prior. This guides the network to suppress unreliable matches in regions with low intensity variation, improving robustness under sparse input. Extensive experiments on KITTI, nuScenes, and MIAS-LCEC-TF70 benchmarks demonstrate that our method achieves state-of-the-art performance, outperforming prior approaches on nuScenes (even those relying on accumulated point clouds), despite using only single-frame LiDAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。