arXiv:2603.02532cs.CV2026-03被引 3

EIMC通过早期协作提升自动驾驶多模态感知效率,显著降低通信开销。

EIMC: Efficient Instance-aware Multi-modal Collaborative Perception

  • 引入轻量级协作体素,在本地融合阶段即注入邻居信息,实现早期协同
  • 在低置信度区域仅传输前K个关键实例向量,达成87.98%带宽节省
  • 基于热图共识机制精准定位需协作区域,适合资源受限的车联网场景

多模态协同感知对提升自动驾驶安全性至关重要。现有方法普遍采用‘局部融合-通信’流程,需高带宽传输个体特征数据以实现协同融合。本文提出EIMC,首创早期协同范式:将邻居传输的轻量级协作体素注入本车本地模态融合步骤,生成紧凑且富含信息的3D协同先验,增强跨模态对齐。随后,通过热图驱动的共识协议,计算像素级置信度热图,精准识别协作需求区域;仅查询该区域内置信度低、差异大的前K个实例向量,经交叉注意力融合完成补全。之后,执行细化融合:收集各智能体置信度最高的前K个实例,利用自注意力增强其特征。该实例中心化消息传递策略有效减少冗余,保障被遮挡关键物体恢复。在OPV2V和DAIR-V2X数据集上,EIMC达到73.01% [email protected],同时相比最优已发表方法降低87.98%字节带宽消耗。代码已开源。

原文摘要 · Abstract (English)

Multi-modal collaborative perception calls for great attention to enhancing the safety of autonomous driving. However, current multi-modal approaches remain a ``local fusion to communication'' sequence, which fuses multi-modal data locally and needs high bandwidth to transmit an individual's feature data before collaborative fusion. EIMC innovatively proposes an early collaborative paradigm. It injects lightweight collaborative voxels, transmitted by neighbor agents, into the ego's local modality-fusion step, yielding compact yet informative 3D collaborative priors that tighten cross-modal alignment. Next, a heatmap-driven consensus protocol identifies exactly where cooperation is needed by computing per-pixel confidence heatmaps. Only the Top-K instance vectors located in these low-confidence, high-discrepancy regions are queried from peers, then fused via cross-attention for completion. Afterwards, we apply a refinement fusion that involves collecting the top-K most confident instances from each agent and enhancing their features using self-attention. The above instance-centric messaging reduces redundancy while guaranteeing that critical occluded objects are recovered. Evaluated on OPV2V and DAIR-V2X, EIMC attains 73.01\% [email protected] while reducing byte bandwidth usage by 87.98\% compared with the best published multi-modal collaborative detector. Code publicly released at https://github.com/sidiangongyuan/EIMC.

多模态感知协同推理带宽优化自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。