通过异构查询交互提升摄像头与雷达融合检测性能
Control Your Queries: Heterogeneous Query Interaction for Camera-Radar Fusion
- 引入图像、雷达和世界查询在3D空间协同初始化
- 跨模态注意力融合使检测精度达67.9 NDS
- 适合自动驾驶多传感器融合研究者参考
在自动驾驶中，摄像头与雷达融合可实现互补感知且部署成本低。现有方法多通过输入拼接、特征图融合或基于查询的特征采样实现融合。本文提出一种新型融合范式——异构查询交互，并构建名为ConFusion的摄像头-雷达3D目标检测器。ConFusion将图像查询、雷达查询及可学习的世界查询分布在3D空间中，以优化查询初始化与目标覆盖范围。为促进异构查询间的交叉交互，引入异构查询混合（QMix），在特征采样后执行专用跨类型注意力，整合互补的目标证据。进一步提出交互式查询交换采样（QSwap），允许相关查询在注意力与几何约束下交换信息丰富的特征标记，提升特征采样质量。在nuScenes数据集上的实验表明，ConFusion达到59.1 mAP和65.6 NDS（验证集），61.6 mAP和67.9 NDS（测试集），性能领先于当前最先进方法。
原文摘要 · Abstract (English)
In autonomous driving, camera-radar fusion offers complementary sensing and low deployment cost. Existing methods perform fusion through input mixing, feature map mixing, or query-based feature sampling. We propose a new fusion paradigm, termed heterogeneous query interaction, and present ConFusion, a camera-radar 3D object detector. ConFusion combines image queries, radar queries, and learnable world queries distributed in 3D space to improve query initialization and object coverage. To encourage cross-type interaction among heterogeneous queries, we introduce heterogeneous query mixing (QMix), which performs dedicated cross-type attention after feature sampling to consolidate complementary object evidence. We further propose interactive query swap sampling (QSwap), which improves feature sampling by allowing related queries to exchange informative feature tokens under attention and geometric constraints. Experiments on the nuScenes dataset show that ConFusion achieves state-of-the-art performance, reaching 59.1 mAP and 65.6 NDS on the validation set, and 61.6 mAP and 67.9 NDS on the test set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。