通过融合环境占位信息,提升视觉3D目标检测的场景理解能力。
Collaborative Perceiver: Elevating Vision-based 3D Object Detection via Local Density-Aware Spatial Occupancy
- 利用局部密度感知的占位标签生成精细环境表征
- 在nuScenes上达到49.5% mAP和59.2% NDS
- 适合关注自动驾驶中视觉3D检测与环境建模融合的研究者
基于视觉的鸟瞰图(BEV)3D目标检测在自动驾驶中因成本低且信息丰富而取得显著进展。然而,现有方法通常通过合并提取的物体特征构建BEV表示,忽视了道路、人行道等内在环境上下文,限制了对物理世界特征的全面感知。为此,我们提出多任务学习框架协同感知器(CoP),利用空间占位作为辅助信息,挖掘3D目标检测与占位预测任务间共享的结构与概念相似性,弥合空间表示与特征优化之间的差距。首先,我们设计一个流程生成包含局部密度信息(LDO)的密集占位真值,以重建详细环境信息;其次,采用体素高度引导采样(VHS)策略,根据物体属性提炼细粒度局部特征;最后,构建全局-局部协同特征融合(CFF)模块,无缝整合两任务互补知识,生成更鲁棒的BEV表示。在nuScenes基准上的大量实验表明,CoP优于现有视觉基框架,在测试集上达到49.5% mAP和59.2% NDS。代码与补充材料见GitHub链接:https://github.com/jichengyuan/Collaborative-Perceiver。
原文摘要 · Abstract (English)
Vision-based bird's-eye-view (BEV) 3D object detection has advanced significantly in autonomous driving by offering cost-effectiveness and rich contextual information. However, existing methods often construct BEV representations by collapsing extracted object features, neglecting intrinsic environmental contexts, such as roads and pavements. This hinders detectors from comprehensively perceiving the characteristics of the physical world. To alleviate this, we introduce a multi-task learning framework, Collaborative Perceiver (CoP), that leverages spatial occupancy as auxiliary information to mine consistent structural and conceptual similarities shared between 3D object detection and occupancy prediction tasks, bridging gaps in spatial representations and feature refinement. To this end, we first propose a pipeline to generate dense occupancy ground truths incorporating local density information (LDO) for reconstructing detailed environmental information. Next, we employ a voxel-height-guided sampling (VHS) strategy to distill fine-grained local features according to distinct object properties. Furthermore, we develop a global-local collaborative feature fusion (CFF) module that seamlessly integrates complementary knowledge between both tasks, thus composing more robust BEV representations. Extensive experiments on the nuScenes benchmark demonstrate that CoP outperforms existing vision-based frameworks, achieving 49.5\% mAP and 59.2\% NDS on the test set. Code and supplementary materials are available at this link https://github.com/jichengyuan/Collaborative-Perceiver.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。