用实例分割引导雷达点云增强,提升单目3D检测的深度估计精度。
Instance-Guided Radar Depth Estimation for 3D Object Detection
- 通过实例分割引导雷达点云扩展,生成更密集且语义对齐的雷达表示。
- 在雷达引导深度估计上达到当前最优,显著提升3D检测性能。
- 适合做单目3D检测中融合雷达信息的科研与工程人员参考。
准确的深度估计是自动驾驶中3D感知的基础,支持检测、跟踪和运动规划等任务。然而,基于单目相机的3D检测存在深度模糊性,在恶劣条件下鲁棒性差。雷达具有抗弱光和恶劣天气的优势,但其稀疏性和低分辨率限制了直接用于检测框架。因此需要更有效的雷达-相机融合策略。本文提出一个端到端框架,包含两个关键组件:首先引入InstaRadar,一种基于实例分割掩码的雷达点云扩展方法,提升雷达密度与语义对齐,生成更结构化的表示,在雷达引导深度估计上达到当前最优;其次将预训练的RCDPT模型替换BEVDepth中的深度模块,并使用InstaRadar增强输入,持续提升3D检测性能。整体优于基线BEVDepth模型,验证了InstaRadar的有效性及显式深度监督的优势。尽管目前框架未直接提取鸟瞰图特征,仍落后于端到端雷达-相机融合模型,提示未来可扩展至点云表示并加入带时序线索的专用雷达分支以增强融合效果。
原文摘要 · Abstract (English)
Accurate depth estimation is fundamental to 3D perception in autonomous driving, supporting tasks such as detection, tracking, and motion planning. However, monocular camera-based 3D detection suffers from depth ambiguity and reduced robustness under challenging conditions. Radar provides complementary advantages such as resilience to poor lighting and adverse weather, but its sparsity and low resolution limit its direct use in detection frameworks. This motivates the need for effective Radar-camera fusion with improved preprocessing and depth estimation strategies. We propose an end-to-end framework that enhances monocular 3D object detection through two key components. First, we introduce InstaRadar, an instance segmentation-guided expansion method that leverages pre-trained segmentation masks to enhance Radar density and semantic alignment, producing a more structured representation. InstaRadar achieves state-of-the-art results in Radar-guided depth estimation, showing its effectiveness in generating high-quality depth features. Second, we integrate the pre-trained RCDPT into the BEVDepth framework as a replacement for its depth module. With InstaRadar-enhanced inputs, the RCDPT integration consistently improves 3D detection performance. Overall, these components yield steady gains over the baseline BEVDepth model, demonstrating the effectiveness of InstaRadar and the advantage of explicit depth supervision in 3D object detection. Although the framework lags behind Radar-camera fusion models that directly extract BEV features, since Radar serves only as guidance rather than an independent feature stream, this limitation highlights potential for improvement. Future work will extend InstaRadar to point cloud-like representations and integrate a dedicated Radar branch with temporal cues for enhanced BEV fusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。