arXiv:2510.19618cs.CV2025-10NeurIPS被引 9

无需重训练,用生成通信实现异构多智能体高效协作感知

Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism

  • 通过生成式通信机制,以轻量级方式生成适配语义空间的特征
  • 在多个数据集上实现更优感知性能,新增智能体成本降低81%
  • 适合需要快速扩展、资源受限的异构车联网系统

多智能体协作通过信息共享提升个体感知能力。然而,在真实场景中,异构智能体间传感器与模型差异导致协作时出现领域鸿沟。现有基于适应与重建的方法因两大缺陷难以实现实用化协作:(1) 编码器或核心模块的侵入式重训练会破坏智能体间的语义一致性;(2) 新智能体接入带来高计算开销,限制可扩展性。为此,我们提出新型生成式通信机制(GenComm),通过特征生成实现异构多智能体间的无缝感知,且不修改原有网络结构,并采用轻量级空间信息数值对齐,以极低开销高效融合新智能体。具体而言,设计可变形消息提取器为每个协作者提取空间消息,替代中间特征进行传输;空间感知特征生成器利用条件扩散模型生成与本体语义空间一致的特征,同时保留协作者的空间信息;生成特征再经通道增强模块优化后融合。在OPV2V-H、DAIR-V2X和V2X-Real数据集上的实验表明,GenComm优于现有最先进方法,在引入新智能体时计算成本与参数量均降低81%。代码已开源。

原文摘要 · Abstract (English)

Multi-agent collaboration enhances the perception capabilities of individual agents through information sharing. However, in real-world applications, differences in sensors and models across heterogeneous agents inevitably lead to domain gaps during collaboration. Existing approaches based on adaptation and reconstruction fail to support pragmatic heterogeneous collaboration due to two key limitations: (1) Intrusive retraining of the encoder or core modules disrupts the established semantic consistency among agents; and (2) accommodating new agents incurs high computational costs, limiting scalability. To address these challenges, we present a novel Generative Communication mechanism (GenComm) that facilitates seamless perception across heterogeneous multi-agent systems through feature generation, without altering the original network, and employs lightweight numerical alignment of spatial information to efficiently integrate new agents at minimal cost. Specifically, a tailored Deformable Message Extractor is designed to extract spatial message for each collaborator, which is then transmitted in place of intermediate features. The Spatial-Aware Feature Generator, utilizing a conditional diffusion model, generates features aligned with the ego agent's semantic space while preserving the spatial information of the collaborators. These generated features are further refined by a Channel Enhancer before fusion. Experiments conducted on the OPV2V-H, DAIR-V2X and V2X-Real datasets demonstrate that GenComm outperforms existing state-of-the-art methods, achieving an 81% reduction in both computational cost and parameter count when incorporating new agents. Our code is available at https://github.com/jeffreychou777/GenComm.

多智能体异构协作生成通信车联网

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。