用强化学习优化无人机网络,减少通信延迟并提升吞吐量。
Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications
- 引入延迟惩罚奖励机制,促进无人机间信息共享
- 实现信息延迟降低50%、吞吐量提升75%
- 适合需要高效协作的无人机无线网络部署
本文利用多架无人机通过中继通信加速地面用户向远程基站传输数据。由于无人机间信息交换存在间歇性,常导致系统状态获取延迟,影响协同效率。为最大化整体吞吐量,提出一种容忍延迟的多智能体深度强化学习(MADRL)算法,结合延迟惩罚奖励以鼓励无人机间信息共享,并联合优化无人机轨迹规划、网络形成与传输控制策略。针对信道不可靠导致的信息丢失问题,进一步提出基于时空注意力的预测方法,恢复丢失信息,增强各无人机对网络状态的认知。仿真结果表明,该方法相较传统MADRL实现信息延迟降低超过50%、吞吐量提升75%。值得注意的是,提升信息共享不会牺牲网络容量,反而显著提高学习性能与吞吐量;同时降低对频繁信息交换的依赖,有利于MADRL在无人机辅助无线网络中的实际部署。
原文摘要 · Abstract (English)
In this paper, we employ multiple UAVs to accelerate data transmissions from ground users (GUs) to a remote base station (BS) via the UAVs' relay communications. The UAVs' intermittent information exchanges typically result in delays in acquiring the complete system state and hinder their effective collaboration. To maximize the overall throughput, we first propose a delay-tolerant multi-agent deep reinforcement learning (MADRL) algorithm that integrates a delay-penalized reward to encourage information sharing among UAVs, while jointly optimizing the UAVs' trajectory planning, network formation, and transmission control strategies. Additionally, considering information loss due to unreliable channel conditions, we further propose a spatio-temporal attention based prediction approach to recover the lost information and enhance each UAV's awareness of the network state. These two designs are envisioned to enhance the network capacity in UAV-assisted wireless networks with limited communications. The simulation results reveal that our new approach achieves over 50\% reduction in information delay and 75% throughput gain compared to the conventional MADRL. Interestingly, it is shown that improving the UAVs' information sharing will not sacrifice the network capacity. Instead, it significantly improves the learning performance and throughput simultaneously. It is also effective in reducing the need for UAVs' information exchange and thus fostering practical deployment of MADRL in UAV-assisted wireless networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。