通过动态专家混合模型,让多智能体感知融合更懂视角差异。
CoBEVMoE: Heterogeneity-aware Feature Fusion with Dynamic Mixture-of-Experts for Collaborative Perception
- 用动态专家网络捕捉各智能体的观测差异
- 在两个数据集上分别提升1.5%和3.0%的检测精度
- 适合需要融合异构感知数据的自动驾驶场景
协同感知通过多智能体间共享信息,拓展感知范围并提升精度。然而,由于视角和空间位置差异,各智能体获取的观测具有异质性。现有中间融合方法多关注特征对齐,忽视了智能体间的感知多样性。为此,我们提出CoBEVMoE,一种基于鸟瞰图(BEV)空间的新型协同感知框架,引入动态专家混合(DMoE)结构。每个专家由特定智能体输入动态生成,既能提取独特可靠特征,又能关注共享语义。该设计使融合过程显式建模跨智能体的特征相似性与异质性。此外,我们提出动态专家度量损失(DEML),增强专家间差异性,提升融合表示的判别能力。在OPV2V和DAIR-V2X-C数据集上的大量实验表明,CoBEVMoE达到当前最优性能:在OPV2V上相机基BEV分割的IoU提升+1.5%,在DAIR-V2X-C上激光雷达基3D目标检测的[email protected]提升+3.0%,验证了基于专家的异质特征建模在多智能体协同感知中的有效性。源代码将公开于https://github.com/godk0509/CoBEVMoE。
原文摘要 · Abstract (English)
Collaborative perception aims to extend sensing coverage and improve perception accuracy by sharing information among multiple agents. However, due to differences in viewpoints and spatial positions, agents often acquire heterogeneous observations. Existing intermediate fusion methods primarily focus on aligning similar features, often overlooking the perceptual diversity among agents. To address this limitation, we propose CoBEVMoE, a novel collaborative perception framework that operates in the Bird's Eye View (BEV) space and incorporates a Dynamic Mixture-of-Experts (DMoE) architecture. In DMoE, each expert is dynamically generated based on the input features of a specific agent, enabling it to extract distinctive and reliable cues while attending to shared semantics. This design allows the fusion process to explicitly model both feature similarity and heterogeneity across agents. Furthermore, we introduce a Dynamic Expert Metric Loss (DEML) to enhance inter-expert diversity and improve the discriminability of the fused representation. Extensive experiments on the OPV2V and DAIR-V2X-C datasets demonstrate that CoBEVMoE achieves state-of-the-art performance. Specifically, it improves the IoU for Camera-based BEV segmentation by +1.5% on OPV2V and the [email protected] for LiDAR-based 3D object detection by +3.0% on DAIR-V2X-C, verifying the effectiveness of expert-based heterogeneous feature modeling in multi-agent collaborative perception. The source code will be made publicly available at https://github.com/godk0509/CoBEVMoE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。