arXiv:2412.06142cs.CVcs.RO2024-12被引 8

解决车联网中多车传感器异构错位问题,提升协同感知鲁棒性。

AgentAlign: Misalignment-Adapted Multi-Agent Perception for Resilient Inter-Agent Sensor Correlations

  • 构建跨模态对齐空间与异构特征对齐机制,动态调和多车感知差异。
  • 在真实数据集上实现最优性能,显著提升复杂环境下的感知一致性。
  • 专为真实世界噪声设计数据集,适合自动驾驶协同感知研究者使用。

协同感知因能利用联网自动驾驶车辆(CAVs)与智能基础设施间的共享信息,有效缓解感知遮挡与范围限制问题,受到广泛关注。然而,现有研究忽视了多智能体环境下多传感器关联的脆弱性:异构传感器测量值极易受环境因素影响,导致车际感知交互减弱。不同运行条件与现实因素不可避免引入多重噪声,造成多传感器错位,使多车多模态感知在真实场景中部署极具挑战。本文提出AgentAlign,一种面向真实世界的异构多智能体跨模态特征对齐框架,以有效应对多模态错位问题。方法引入跨模态特征对齐空间(CFAS)与异构智能体特征对齐(HAFA)机制,动态调和各智能体间多模态特征。同时,我们构建了新颖的V2XSet-noise数据集,模拟多样环境下的真实传感器缺陷,系统评估所提方法的鲁棒性。在V2X-Real与V2XSet-Noise基准上的大量实验表明,本框架达到当前最优性能,凸显其在真实协同自动驾驶中的应用潜力。可控的V2XSet-Noise数据集及生成管道将后续公开。

原文摘要 · Abstract (English)

Cooperative perception has attracted wide attention given its capability to leverage shared information across connected automated vehicles (CAVs) and smart infrastructures to address sensing occlusion and range limitation issues. However, existing research overlooks the fragile multi-sensor correlations in multi-agent settings, as the heterogeneous agent sensor measurements are highly susceptible to environmental factors, leading to weakened inter-agent sensor interactions. The varying operational conditions and other real-world factors inevitably introduce multifactorial noise and consequentially lead to multi-sensor misalignment, making the deployment of multi-agent multi-modality perception particularly challenging in the real world. In this paper, we propose AgentAlign, a real-world heterogeneous agent cross-modality feature alignment framework, to effectively address these multi-modality misalignment issues. Our method introduces a cross-modality feature alignment space (CFAS) and heterogeneous agent feature alignment (HAFA) mechanism to harmonize multi-modality features across various agents dynamically. Additionally, we present a novel V2XSet-noise dataset that simulates realistic sensor imperfections under diverse environmental conditions, facilitating a systematic evaluation of our approach's robustness. Extensive experiments on the V2X-Real and V2XSet-Noise benchmarks demonstrate that our framework achieves state-of-the-art performance, underscoring its potential for real-world applications in cooperative autonomous driving. The controllable V2XSet-Noise dataset and generation pipeline will be released in the future.

协同感知多模态对齐自动驾驶车联网

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。