arXiv:2512.12884cs.CVcs.RO2025-12中稿 · IV2025

用Transformer融合摄像头图像与传感器对象列表,提升3D目标检测精度。

Cross-Level Sensor Fusion with Object Lists via Transformer for 3D Object Detection

  • 将抽象对象列表作为去噪查询输入Transformer,实现跨层级融合。
  • 在nuScenes上比纯视觉基线提升显著,且对噪声有良好鲁棒性。
  • 首次提出跨层级融合框架,适合自动驾驶多源感知系统研究者。

在智能汽车传感器融合系统中,常用智能传感器和车联网(V2X)模块,其数据通常以处理后的对象列表形式存在,而非原始传感器数据。为避免分别处理原始数据再于对象层面融合的繁琐流程,本文提出一种基于Transformer的端到端跨层级融合方法,将高抽象级别的对象列表信息与原始相机图像融合,用于3D目标检测。对象列表作为去噪查询输入Transformer,与可学习查询一同参与后续特征聚合。此外,引入由对象列表位置与尺寸先验生成的可变形高斯掩码,显式引导注意力至感兴趣区域,加速模型训练收敛。由于目前尚无公开的仅含对象列表的基准数据集,我们提出一种方法:通过模拟状态噪声及误检/漏检,从真实边界框生成伪对象列表。作为首个开展跨层级融合的工作,该方法在nuScenes数据集上显著优于纯视觉基线,且在不同噪声水平的模拟对象列表及真实探测器输出上均表现出良好的泛化能力。

原文摘要 · Abstract (English)

In automotive sensor fusion systems, smart sensors and Vehicle-to-Everything (V2X) modules are commonly utilized. Sensor data from these systems are typically available only as processed object lists rather than raw sensor data from traditional sensors. Instead of processing other raw data separately and then fusing them at the object level, we propose an end-to-end cross-level fusion concept with Transformer, which integrates highly abstract object list information with raw camera images for 3D object detection. Object lists are fed into a Transformer as denoising queries and propagated together with learnable queries through the latter feature aggregation process. Additionally, a deformable Gaussian mask, derived from the positional and size dimensional priors from the object lists, is explicitly integrated into the Transformer decoder. This directs attention toward the target area of interest and accelerates model training convergence. Furthermore, as there is no public dataset containing object lists as a standalone modality, we propose an approach to generate pseudo object lists from ground-truth bounding boxes by simulating state noise and false positives and negatives. As the first work to conduct cross-level fusion, our approach shows substantial performance improvements over the vision-based baseline on the nuScenes dataset. It demonstrates its generalization capability over diverse noise levels of simulated object lists and real detectors.

3D检测传感器融合Transformer自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。