arXiv:2412.18884cs.CV2024-12被引 8

提出分层采样方法,提升多视角3D目标检测的垂直定位精度。

HV-BEV: Decoupling Horizontal and Vertical Feature Sampling for Multi-View 3D Object Detection

  • 将水平与垂直特征采样解耦,增强跨视图关联性
  • 在nuScenes上达50.5% mAP、59.8% NDS,性能领先
  • 适合需要高精度高度感知的自动驾驶感知系统

视觉多视角环境感知在自动驾驶中日益重要,尤其是基于鸟瞰图(BEV)的模型。现有先进方法主要通过显式或隐式深度预测将各摄像头图像特征编码到BEV空间,但常忽略物体在三维空间中的结构相关性及不同类别物体占据的局部高度范围差异。例如卡车位于较高处,而交通锥靠近地面。本文提出一种新方法——HV-BEV,将BEV网格查询中的特征采样解耦为水平特征聚合与垂直自适应高度感知参考点采样。具体而言,针对地平面每个3D参考点动态构建邻近点集,增强同一实例在多个BEV网格间的关联性,尤其适用于跨多视角覆盖的物体。此外,摒弃固定高度范围的均匀采样,引入结合历史信息的高度感知模块,使参考点能自适应聚焦于不同场景中物体出现的实际高度。大量实验验证了该方法的有效性,在nuScenes数据集上表现优于基线模型。最优模型在nuScenes测试集上达到50.5% mAP和59.8% NDS。代码已开源。

原文摘要 · Abstract (English)

The application of vision-based multi-view environmental perception system has been increasingly recognized in autonomous driving technology, especially the BEV-based models. Current state-of-the-art solutions primarily encode image features from each camera view into the BEV space through explicit or implicit depth prediction. However, these methods often overlook the structured correlations among different parts of objects in 3D space and the fact that different categories of objects often occupy distinct local height ranges. For example, trucks appear at higher elevations, whereas traffic cones are near the ground. In this work, we propose a novel approach that decouples feature sampling in the \textbf{BEV} grid queries paradigm into \textbf{H}orizontal feature aggregation and \textbf{V}ertical adaptive height-aware reference point sampling (HV-BEV), aiming to improve both the aggregation of objects' complete information and awareness of diverse objects' height distribution. Specifically, a set of relevant neighboring points is dynamically constructed for each 3D reference point on the ground-aligned horizontal plane, enhancing the association of the same instance across different BEV grids, especially when the instance spans multiple image views around the vehicle. Additionally, instead of relying on uniform sampling within a fixed height range, we introduce a height-aware module that incorporates historical information, enabling the reference points to adaptively focus on the varying heights at which objects appear in different scenes. Extensive experiments validate the effectiveness of our proposed method, demonstrating its superior performance over the baseline across the nuScenes dataset. Moreover, our best-performing model achieves a remarkable 50.5\% mAP and 59.8\% NDS on the nuScenes testing set. The code is available at https://github.com/Uddd821/HV-BEV.

3D检测鸟瞰图自动驾驶多视角

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。