用大模型压缩无人机视觉数据,85%减通信量,感知性能提升5%以上。
Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks
- 选关键像素稀疏传输,降低数据量和延迟
- 基于Swin-large的BEV特征融合,提升车辆感知效果
- 用扩散强化学习联合优化飞行、压缩与波束成形
多无人飞行器(UAV)协同感知已成为低空经济应用的有力范式,通过无线通信整合多视角观测以提升感知性能。然而,多架无人机产生的海量视觉数据对通信延迟和资源效率带来严峻挑战。为此,本文提出一种高效通信的协同感知框架——基站辅助无人机(BHU),在降低通信开销的同时提升感知表现。具体地,采用Top-K选择机制提取无人机捕获的RGB图像中最具信息量的像素,实现视觉数据稀疏化传输,显著减少数据体积与延迟;稀疏图像通过多用户MIMO(MU-MIMO)传至地面服务器,由基于Swin-large的MaskDINO编码器提取鸟瞰图(BEV)特征并完成协同特征融合,用于地面车辆感知。此外,设计基于扩散模型的深度强化学习(DRL)算法,联合优化协作无人机选择、稀疏化比率与预编码矩阵,在通信效率与感知效用间取得平衡。在Air-Co-Pred数据集上的仿真结果表明,相较于传统基于CNN的BEV融合基线,所提BHU框架感知性能提升超5%,通信开销降低85%,为资源受限无线环境下的多无人机协同感知提供了有效解决方案。
原文摘要 · Abstract (English)
Multi-uncrewed aerial vehicle (UAV) cooperative perception has emerged as a promising paradigm for diverse low-altitude economy applications, where complementary multi-view observations are leveraged to enhance perception performance via wireless communications. However, the massive visual data generated by multiple UAVs poses significant challenges in terms of communication latency and resource efficiency. To address these challenges, this paper proposes a communication-efficient cooperative perception framework, termed Base-Station-Helped UAV (BHU), which reduces communication overhead while enhancing perception performance. Specifically, we employ a Top-K selection mechanism to identify the most informative pixels from UAV-captured RGB images, enabling sparsified visual transmission with reduced data volume and latency. The sparsified images are transmitted to a ground server via multi-user MIMO (MU-MIMO), where a Swin-large-based MaskDINO encoder extracts bird's-eye-view (BEV) features and performs cooperative feature fusion for ground vehicle perception. Furthermore, we develop a diffusion model-based deep reinforcement learning (DRL) algorithm to jointly select cooperative UAVs, sparsification ratios, and precoding matrices, achieving a balance between communication efficiency and perception utility. Simulation results on the Air-Co-Pred dataset demonstrate that, compared with traditional CNN-based BEV fusion baselines, the proposed BHU framework improves perception performance by over 5% while reducing communication overhead by 85%, providing an effective solution for multi-UAV cooperative perception under resource-constrained wireless environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。