用先前地图先验提升摄像头3D目标检测精度
Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Tracking

- 双空间融合:将地图投影到视角图与鸟瞰图,联合增强图像特征
- 在nuScenes和Argoverse 2上显著提升定位精度,尤其在无LiDAR时表现优异
- 适合部署在重复路径的自动驾驶场景,无需额外传感器
基于摄像头的3D目标检测与跟踪是自动驾驶的核心技术,但缺乏深度信息导致定位不准确,尤其在未配备昂贵在线LiDAR的情况下。然而,在许多实际部署中,车辆会反复经过相同环境,因此可利用之前遍历生成的静态点云地图作为几何先验。本文提出DualViewMapDet,一种仅依赖摄像头的推理框架,可在线检索并利用这些地图先验来弥补无LiDAR的不足。核心思想是双空间相机-地图融合策略,避免单向视角转换。具体包括:(i) 将地图投影至透视视图(PV),编码多通道几何线索以增强图像特征并支持鸟瞰图(BEV)重构;(ii) 使用稀疏体素骨干网络将地图直接编码为鸟瞰图,并在共享度量空间中与提升后的相机特征融合。在nuScenes和Argoverse 2上的大量实验表明,该方法持续优于强基线相机模型,尤其在目标定位方面提升显著。消融实验进一步验证了PV/BEV融合与地图覆盖范围的有效性。代码与预训练模型已公开于 https://dualviewmapdet.cs.uni-freiburg.de。
原文摘要 · Abstract (English)
Camera-based 3D object detection and tracking are central to autonomous driving, yet precise 3D object localization remains fundamentally constrained by depth ambiguity when no expensive, depth-rich online LiDAR is available at inference. In many deployments, however, vehicles repeatedly traverse the same environments, making static point cloud maps from prior traversals a practical source of geometric priors. We propose DualViewMapDet, a camera-only inference framework that retrieves such map priors online and leverages them to mitigate the absence of a LiDAR sensor during deployment. The key idea is a dual-space camera-map fusion strategy that avoids one-sided view conversion. Specifically, we (i) project the map into perspective view (PV) and encode multi-channel geometric cues to enrich image features and support BEV lifting, and (ii) encode the map directly in bird's-eye view (BEV) with a sparse voxel backbone and fuse it with lifted camera features in a shared metric space. Extensive evaluations on nuScenes and Argoverse 2 demonstrate consistent improvements over strong camera-only baselines, with particularly strong gains in object localization. Ablations further validate the contributions of PV/BEV fusion and prior-map coverage. We make the code and pre-trained models available at https://dualviewmapdet.cs.uni-freiburg.de .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。