用雷达补全与序列解码提升雷达相机3D检测精度
RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection
- 通过雷达稠密化增强稀疏点云，融合图像特征提升信息完整度
- 在nuScenes上达到新最好效果，81.6%的mAP，比前序方法高2.3%
- 适合做多模态自动驾驶感知，尤其关注雷达信号弱场景
在雷达-相机3D目标检测中，雷达点云稀疏且噪声大，导致模态融合困难。为此，我们提出一种基于查询的新检测方法RCTrans。首先设计雷达稠密编码器，丰富稀疏的有效雷达令牌，并将其与图像令牌拼接，从而充分挖掘每个感兴趣区域的3D信息，减少空令牌在融合阶段的干扰。随后设计剪枝序列解码器，基于获得的令牌和随机初始化查询预测3D边界框。为缓解雷达点云中的俯仰模糊问题，采用渐进式融合结构逐步定位目标，提升令牌与查询间对应关系的精确性与灵活性。解码器采用剪枝训练策略，显著降低推理时间并防止查询失去区分性。在大规模nuScenes数据集上的大量实验验证了该方法的优势，取得新的最优结果，模型代码已公开于https://github.com/liyih/RCTrans。
原文摘要 · Abstract (English)
In radar-camera 3D object detection, the radar point clouds are sparse and noisy, which causes difficulties in fusing camera and radar modalities. To solve this, we introduce a novel query-based detection method named Radar-Camera Transformer (RCTrans). Specifically, we first design a Radar Dense Encoder to enrich the sparse valid radar tokens, and then concatenate them with the image tokens. By doing this, we can fully explore the 3D information of each interest region and reduce the interference of empty tokens during the fusing stage. We then design a Pruning Sequential Decoder to predict 3D boxes based on the obtained tokens and random initialized queries. To alleviate the effect of elevation ambiguity in radar point clouds, we gradually locate the position of the object via a sequential fusion structure. It helps to get more precise and flexible correspondences between tokens and queries. A pruning training strategy is adopted in the decoder, which can save much time during inference and inhibit queries from losing their distinctiveness. Extensive experiments on the large-scale nuScenes dataset prove the superiority of our method, and we also achieve new state-of-the-art radar-camera 3D detection results. Our implementation is available at https://github.com/liyih/RCTrans.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。