针对恶劣天气下3D目标检测难题,提出可自适应选择专家的混合专家框架。
AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection
- 基于图像特征设计天气感知路由,精准识别天气类型并激活相关专家。
- 在真实数据集上实现恶劣天气检测性能提升约15%,推理开销几乎不变。
- 适用于多种主流检测器,具备强可扩展性,适合自动驾驶场景应用。
恶劣天气下的鲁棒3D目标检测对自动驾驶至关重要。然而,现有方法通常简单混合不同天气样本训练,忽视了各天气场景间的数据分布差异,导致性能冲突。为此,本文提出AW-MoE框架,创新性地将混合专家(MoE)引入天气鲁棒的多模态3D目标检测。AW-MoE采用图像引导的天气感知路由(IWR),利用图像特征在不同天气下的强区分性及其对场景变化的不变性,实现精确天气分类。基于此分类结果,IWR选择前K个最相关的天气特异性专家(WSE)处理数据偏差,确保各类天气下的最优检测。此外,提出统一双模态增强(UDMA),同步进行激光雷达与4D雷达数据增强,保持场景真实感。大量实验证明,AW-MoE在真实数据集上相比当前最优方法,恶劣天气性能提升约15%,且推理开销极低。将AW-MoE集成至现有基准检测器,性能超越现有最先进水平。结果表明该方法有效且具备强可扩展性。代码将公开于https://github.com/windlinsherlock/AW-MoE。
原文摘要 · Abstract (English)
Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。