用视觉大模型增强雷达感知，提升恶劣天气下多类目标检测能力
DinoRADE: Full Spectral Radar-Camera Fusion with Vision Foundation Model Features for Multi-class Object Detection in Adverse Weather
- 以雷达为中心，通过可变形交叉注意力融合相机视觉特征
- 在K-Radar数据集上实现12.1%性能提升，首次报告五类目标独立检测结果
- 特别适合小目标和脆弱道路使用者的恶劣天气检测场景
可靠的天气鲁棒感知系统对自动驾驶安全至关重要，通常依赖多模态传感器实现全面环境感知。尽管现有基于FMCW雷达的方法在恶劣天气下检测任务中表现优异，但在解析细粒度空间细节方面仍存在局限，尤其影响对小型及脆弱道路使用者（VRUs）的识别。此外，现有研究尚未充分解决在如K-Radar等恶劣天气数据集上的VRU检测问题。本文提出DinoRADE，一种以雷达为核心的检测流水线，处理密集雷达张量，并通过可变形交叉注意力在相机视角下聚合变换参考点周围的视觉特征，视觉特征由DINOv3视觉基础模型提供。我们在K-Radar数据集上进行了全面性能评估，涵盖所有天气条件，并首次报告了五类目标的独立检测性能。与现有单类检测方法相比，我们的方法在多类检测任务中优于近期雷达-相机融合方法12.1%。代码已开源。
原文摘要 · Abstract (English)
Reliable and weather-robust perception systems are essential for safe autonomous driving and typically employ multi-modal sensor configurations to achieve comprehensive environmental awareness. While recent automotive FMCW Radar-based approaches achieved remarkable performance on detection tasks in adverse weather conditions, they exhibited limitations in resolving fine-grained spatial details particularly critical for detecting smaller and vulnerable road users (VRUs). Furthermore, existing research has not adequately addressed VRU detection in adverse weather datasets such as K-Radar. We present DinoRADE, a Radar-centered detection pipeline that processes dense Radar tensors and aggregates vision features around transformed reference points in the camera perspective via deformable cross-attention. Vision features are provided by a DINOv3 Vision Foundation Model. We present a comprehensive performance evaluation on the K-Radar dataset in all weather conditions and are among the first to report detection performance individually for five object classes. Additionally, we compare our method with existing single-class detection approaches and outperform recent Radar-camera approaches by 12.1%. The code is available under https://github.com/chr-is-tof/RADE-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。