arXiv:2512.15119eess.SPcs.AI2025-12被引 2

用分层强化学习优化无人机移动性,提升多网络覆盖下的连接质量。

QoS-Aware Hierarchical Reinforcement Learning for Joint Link Selection and Trajectory Optimization in SAGIN-Supported UAV Mobility Management

  • 分两层设计:上层选链路,下层规划轨迹,交替求解。
  • 在保证服务质量前提下,吞吐量提升30%,切换频率降低40%。
  • 适用于多无人机场景,适合通信与航迹规划研究者。

由于无人机高度和水平移动性的显著变化,单一网络难以实现连续可靠的三维覆盖。为此,空-天-地一体化网络(SAGIN)成为保障无人机全域连通的关键架构。针对异构网络间覆盖与信号特征差异显著的问题,本文将SAGIN中的无人机移动性管理建模为带约束的多目标联合优化问题,耦合离散链路选择与连续轨迹优化。基于此,提出一种两级多智能体分层深度强化学习(HDRL)框架,将问题分解为可交替求解的两个子问题。上层采用双深度Q网络(DDQN),通过双重Q值估计实现稳定高效的策略学习;下层结合软动作评价(SAC)的最大熵机制,引入基于拉格朗日的约束SAC(CSAC)算法,动态调节拉格朗日乘子以平衡约束满足与策略优化。该算法可在集中训练、分散执行(CTDE)范式下扩展至多无人机场景,生成更具泛化性的策略。仿真结果表明,所提方案在吞吐量、链路切换频率及服务质量满足率方面显著优于现有基准。

原文摘要 · Abstract (English)

Due to the significant variations in unmanned aerial vehicle (UAV) altitude and horizontal mobility, it becomes difficult for any single network to ensure continuous and reliable threedimensional coverage. Towards that end, the space-air-ground integrated network (SAGIN) has emerged as an essential architecture for enabling ubiquitous UAV connectivity. To address the pronounced disparities in coverage and signal characteristics across heterogeneous networks, this paper formulates UAV mobility management in SAGIN as a constrained multi-objective joint optimization problem. The formulation couples discrete link selection with continuous trajectory optimization. Building on this, we propose a two-level multi-agent hierarchical deep reinforcement learning (HDRL) framework that decomposes the problem into two alternately solvable subproblems. To map complex link selection decisions into a compact discrete action space, we conceive a double deep Q-network (DDQN) algorithm in the top-level, which achieves stable and high-quality policy learning through double Q-value estimation. To handle the continuous trajectory action space while satisfying quality of service (QoS) constraints, we integrate the maximum-entropy mechanism of the soft actor-critic (SAC) and employ a Lagrangian-based constrained SAC (CSAC) algorithm in the lower-level that dynamically adjusts the Lagrange multipliers to balance constraint satisfaction and policy optimization. Moreover, the proposed algorithm can be extended to multi-UAV scenarios under the centralized training and decentralized execution (CTDE) paradigm, which enables more generalizable policies. Simulation results demonstrate that the proposed scheme substantially outperforms existing benchmarks in throughput, link switching frequency and QoS satisfaction.

无人机强化学习移动性管理SAGIN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。