提出首个基于伪图像的两阶段3D目标检测框架,兼顾速度与精度。
3DPillars: Pillar-based two-stage 3D object detection
- 设计3DPillars网络,用2D卷积从伪图像中高效提取3D体素特征。
- 在KITTI和Waymo数据集上达到领先精度,速度优于多数两阶段方法。
- 适合追求高精度且需实时推理的自动驾驶3D检测场景。
PointPillars是最快的一种3D目标检测器,利用伪图像表示来编码场景中3D物体的特征。尽管高效,但其性能通常落后于先进方法,主要受限于:1)伪图像表示无法精确保留3D结构;2)难以采用通常更优的两阶段检测流程(基于3D候选框)。本文首次提出基于伪图像表示的两阶段3D检测框架,缩小了PointPillars与先进方法之间的性能差距,同时保持高效性。框架包含两个新组件:首先,提出3DPillars网络,通过可分离的体素特征模块,仅使用2D卷积从伪图像中高效学习3D体素特征;其核心思想是将3D体素特征视为一系列伪图像堆叠。其次,引入带有稀疏场景上下文特征模块的RoI头,聚合3DPillars的多尺度特征以获得稀疏场景特征,从而有效支持两阶段流程,并充分利用场景上下文信息精炼3D候选框。在KITTI和Waymo Open数据集上的实验表明,该方法在速度与精度之间取得良好平衡,显著提升性能。
原文摘要 · Abstract (English)
PointPillars is the fastest 3D object detector that exploits pseudo image representations to encode features for 3D objects in a scene. Albeit efficient, PointPillars is typically outperformed by state-of-the-art 3D detection methods due to the following limitations: 1) The pseudo image representations fail to preserve precise 3D structures, and 2) they make it difficult to adopt a two-stage detection pipeline using 3D object proposals that typically shows better performance than a single-stage approach. We introduce in this paper the first two-stage 3D detection framework exploiting pseudo image representations, narrowing the performance gaps between PointPillars and state-of-the-art methods, while retaining its efficiency. Our framework consists of two novel components that overcome the aforementioned limitations of PointPillars: First, we introduce a new CNN architecture, dubbed 3DPillars, that enables learning 3D voxel-based features from the pseudo image representation efficiently using 2D convolutions. The basic idea behind 3DPillars is that 3D features from voxels can be viewed as a stack of pseudo images. To implement this idea, we propose a separable voxel feature module that extracts voxel-based features without using 3D convolutions. Second, we introduce an RoI head with a sparse scene context feature module that aggregates multi-scale features from 3DPillars to obtain a sparse scene feature. This enables adopting a two-stage pipeline effectively, and fully leveraging contextual information of a scene to refine 3D object proposals. Experimental results on the KITTI and Waymo Open datasets demonstrate the effectiveness and efficiency of our approach, achieving a good compromise in terms of speed and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。