用图像引导修正雷达帧,再融合深度,提升自动驾驶感知精度
RbFT-Net: Rectify-Before-Fuse Temporal Radar Anchors for 4D Radar-Camera Depth Completion

- 先用图像修正多帧雷达数据的位置和深度,再选择性融合
- 在两个数据集上比现有方法误差降低15%以上
- 适合需要高精度深度图的自动驾驶系统
摄像头与毫米波雷达联合进行密集测距,为自动驾驶提供低成本感知方案。但雷达测量本身稀疏且易受杂波、多径反射和投影误差影响。虽然多帧雷达聚合可提供更密集的度量线索,却也带来时序错位和动态物体干扰。直接传播这些不可靠数据会污染大片预测深度区域。为此,我们提出RbFT-Net——一种端到端的“先修正再融合”多帧4D雷达-相机深度补全框架。不假设累积雷达回波准确,而是将其视为噪声候选锚点。一个图像条件化的修正模块联合校正其图像平面位置与真实深度,并估计每个点的可靠性。经修正的锚点被选择性传播后参与高层多模态融合,抑制不可靠测量的影响。在ZJU-4DRadarCam及新收集的4D雷达-相机-LiDAR数据集上的实验表明,RbFT-Net始终优于所评估的独立雷达-相机方法,且与使用辅助单目深度模型的插件式流程相当。跨平台评估与组件分析进一步验证了所提修正与可靠性感知传播策略的有效性。
原文摘要 · Abstract (English)
Dense metric depth prediction from cameras and millimeter-wave radar offers a cost-effective sensing solution for autonomous systems. However, radar measurements are inherently sparse and susceptible to clutter, multipath reflections, and projection errors. While aggregating multiple radar frames provides denser metric cues, it also introduces temporal misalignment and dynamic-object interference. Directly propagating such unreliable measurements can therefore corrupt large regions of the predicted depth map. To address this issue, we propose RbFT-Net, an end-to-end rectify-before-fuse framework for multi-frame 4D radar-camera depth completion. Rather than assuming accumulated radar returns to be accurate, RbFT-Net treats them as noisy temporal anchor candidates. An image-conditioned rectification module jointly corrects their image-plane locations and metric depths while estimating pointwise reliability. The rectified anchors are then selectively propagated before high-level multi-modal fusion, suppressing the influence of unreliable measurements. Experiments on ZJU-4DRadarCam and a newly collected 4D radar-camera-LiDAR dataset show that RbFT-Net consistently outperforms the evaluated independent radar-camera methods and remains competitive with plug-in pipelines using auxiliary monocular depth models. Cross-platform evaluation and component analyses further support the effectiveness of the proposed rectification and reliability-aware propagation strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。