融合雷达、图像与文本，提升恶劣天气下深度估计精度。
TRIDE: A Text-assisted Radar-Image weather-aware fusion network for Depth Estimation
- 引入文本辅助的雷达-图像融合机制，增强单目深度估计。
- 在nuScenes上实现MAE降低12.87%、RMSE降低9.08%。
- 支持天气自适应调整雷达权重，适合自动驾驶场景应用。
深度估计对自动驾驶至关重要，旨在解析车辆周围的三维环境。雷达传感器因其成本低且抗干扰能力强，推动了雷达-相机融合方案的发展。然而，现有方法在融合多模态特征时未考虑天气影响，尽管雷达在恶劣天气下比摄像头更稳定。同时，视觉-语言模型虽快速发展，但将语言描述用于深度估计仍具挑战。本文首次提出一种文本生成策略及特征提取融合技术，可提升多种算法在KITTI数据集上的精度。在此基础上，提出TRIDE，通过融合雷达点云信息改进文本特征提取，并设计天气感知融合模块，根据实时天气动态调整雷达权重。在nuScenes数据集上，本方法性能超越现有最优方案，实现MAE下降12.87%、RMSE下降9.08%。代码已开源。
原文摘要 · Abstract (English)
Depth estimation, essential for autonomous driving, seeks to interpret the 3D environment surrounding vehicles. The development of radar sensors, known for their cost-efficiency and robustness, has spurred interest in radar-camera fusion-based solutions. However, existing algorithms fuse features from these modalities without accounting for weather conditions, despite radars being known to be more robust than cameras under adverse weather. Additionally, while Vision-Language models have seen rapid advancement, utilizing language descriptions alongside other modalities for depth estimation remains an open challenge. This paper first introduces a text-generation strategy along with feature extraction and fusion techniques that can assist monocular depth estimation pipelines, leading to improved accuracy across different algorithms on the KITTI dataset. Building on this, we propose TRIDE, a radar-camera fusion algorithm that enhances text feature extraction by incorporating radar point information. To address the impact of weather on sensor performance, we introduce a weather-aware fusion block that adaptively adjusts radar weighting based on current weather conditions. Our method, benchmarked on the nuScenes dataset, demonstrates performance gains over the state-of-the-art, achieving a 12.87% improvement in MAE and a 9.08% improvement in RMSE. Code: https://github.com/harborsarah/TRIDE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。