一次推理同时完成多视角2D与3D目标检测，提升自动驾驶感知效率。
SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Cameras
- 统一2D与3D检测为端到端模型，通过交互式解码器实现双向优化。
- 在nuScenes上2D/3D检测均达顶尖性能，长距检测可达150米。
- 支持混合标注数据，降低对昂贵3D标签的依赖，适合实际部署。
多摄像头自动驾驶中，同时感知透视视图下的2D物体和鸟瞰图（BEV）中的3D物体极具挑战。现有两阶段方法仅将2D结果作为一次性线索用于3D检测。本文提出SimPB++，从多摄像头图像中同时检测2D物体（透视视图）与3D物体（鸟瞰视图）。该模型将两类任务统一为端到端架构，采用混合解码器结构，动态耦合多视角2D与3D解码器。引入两个新模块：动态查询分配（Dynamic Query Allocation）自适应地将2D查询分配给3D候选对象；自适应查询聚合（Adaptive Query Aggregation）利用多视角2D特征优化3D表示，形成3D-2D-3D循环精炼。针对多视角2D检测，采用查询组注意力（Query-group Attention）促进组内通信。设计了裁剪与缩放策略以增强远距离感知，并引入传播去噪策略配合辅助区域建议检测器。模型支持混合监督，可使用仅含2D标注与全标注数据，减少对高成本3D标注的依赖。实验表明，SimPB++在nuScenes数据集上两项任务均达到当前最优表现，在Argoverse2上实现长达150米的强远距离检测能力。
原文摘要 · Abstract (English)
Simultaneous perception of 2D objects in perspective view and 3D objects in Bird's Eye View (BEV) is challenging for multi-camera autonomous driving. Existing two-stage pipelines use 2D results only as a one-time cue for 3D detection. We propose SimPB++, which simultaneously detects 2D objects in perspective and 3D objects in BEV from multiple cameras. It unifies both tasks into an end-to-end model with a hybrid decoder architecture, coupling multi-view 2D and 3D decoders interactively. Two novel modules enable deep interaction: Dynamic Query Allocation adaptively assigns 2D queries to 3D candidates, and Adaptive Query Aggregation refines 3D representations using multi-view 2D features, forming a cyclic 3D-2D-3D refinement. For multi-view 2D detection, we use Query-group Attention for intra-group communication. We also design a Crop-and-Scale strategy for long-range perception and a Propagating Denoising strategy with an auxiliary RoI detector. SimPB++ supports mixed supervision with 2D-only and fully annotated data, reducing reliance on expensive 3D labels. Experiments show state-of-the-art performance on nuScenes for both tasks and strong long-range detection (up to 150m) on Argoverse2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。