arXiv:2606.01757cs.CV2026-06

用YOLO骨架和RT-DETR头,实现高效实时3D目标检测。

PillarDETR: YOLO-Backbone and RT-DETR Head for Real-Time 3D Object Detection

  • 以YOLOv8的CSP网络替代传统卷积主干,提升特征提取能力。
  • 在KITTI和nuScenes上达到高mAP与低延迟的平衡,优于PointPillars基线。
  • 无需NMS,直接输出3D框,适合自动驾驶等实时场景。

实时3D目标检测是自动驾驶与机器人安全运行的关键。尽管激光雷达点云提供精确空间信息,但高效处理仍具挑战。传统方法依赖复杂的3D卷积或基于锚框的范式,在精度与速度间难以平衡。本文提出PillarDETR,一种融合柱状点云编码与现代2D视觉模型表征力的端到端架构。具体地,PillarDETR采用源自YOLOv8的跨阶段部分(CSP)网络替换标准卷积主干,从伪图像中提取更丰富特征;同时摒弃传统的锚框或中心点检测头,改用实时检测变压器(RT-DETR)解码器。该混合设计使网络能捕捉全局上下文,并直接预测3D边界框,无需非极大值抑制(NMS)。在KITTI与nuScenes数据集上的大量实验表明,PillarDETR实现了良好的平均精度(mAP)与推理延迟权衡。消融实验确认,引入YOLOv8主干与RT-DETR头相较PointPillars基线有显著提升,确立了其作为实时3D感知高效解决方案的地位。

原文摘要 · Abstract (English)

Real-time 3D object detection is a critical component for the safe operation of autonomous driving systems and robotics. While LiDAR point clouds provide accurate spatial information, processing them efficiently remains a significant challenge. Traditional methods rely on complex 3D convolutions or anchor-based paradigms that struggle to balance detection accuracy with inference speed. In this paper, we propose PillarDETR, a novel end-to-end 3D object detection architecture that combines the efficiency of pillar-based LiDAR encoding with the representational power of modern 2D vision models. Specifically, PillarDETR replaces standard convolutional backbones with a Cross Stage Partial (CSP) network derived from YOLOv8, enabling richer feature extraction from pseudoimages. Furthermore, we discard conventional anchor-based or center-based detection heads in favor of a Real-Time Detection Transformer (RT-DETR) decoder. This hybrid design allows the network to capture global context and directly predict 3D bounding boxes without relying on non-maximum suppression (NMS). Extensive experiments on the KITTI and nuScenes benchmarks demonstrate that PillarDETR achieves a compelling trade-off between mean Average Precision (mAP) and inference latency. Our ablation studies confirm that integrating the YOLOv8 backbone and RT-DETR head yields substantial improvements over the PointPillars baseline, establishing PillarDETR as a highly effective solution for real-time 3D perception.

3D检测实时感知目标检测点云处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。