arXiv:2505.12902eess.SYcs.LG2025-05被引 5

用图神经网络强化学习优化设备直连网络的功率分配,降低延迟并保证公平性。

Power Allocation for Delay Optimization in Device-to-Device Networks: A Graph Reinforcement Learning Approach

  • 基于图神经网络的集中式强化学习,融合信道状态与队列信息
  • 平均延迟显著降低,相比基线方法提升超20%,且用户公平性更好
  • 适合需要动态资源调度的D2D通信系统研究者参考

无线通信中追求速率最大化常面临用户公平性挑战。本文提出一种基于图神经网络(GNN)的强化学习(RL)功率分配方法,用于设备到设备(D2D)通信中的延迟优化。该方法不仅考虑信道状态信息,还整合了包延迟、积压包数和已传输包数等状态因子。采用集中式强化学习框架,由中央控制器作为智能体,使用近端策略优化(PPO)算法进行训练。为充分利用网络拓扑信息并增强泛化能力,在PPO的策略网络(actor)和价值网络(critic)中嵌入GNN层,实现状态信息的低维嵌入表示与高效参数更新。仿真结果表明,所提方法能有效降低平均延迟,优于基线方法,具备良好可扩展性与泛化能力。

原文摘要 · Abstract (English)

The pursuit of rate maximization in wireless communication frequently encounters substantial challenges associated with user fairness. This paper addresses these challenges by exploring a novel power allocation approach for delay optimization, utilizing graph neural networks (GNNs)-based reinforcement learning (RL) in device-to-device (D2D) communication. The proposed approach incorporates not only channel state information but also factors such as packet delay, the number of backlogged packets, and the number of transmitted packets into the components of the state information. We adopt a centralized RL method, where a central controller collects and processes the state information. The central controller functions as an agent trained using the proximal policy optimization (PPO) algorithm. To better utilize topology information in the communication network and enhance the generalization of the proposed method, we embed GNN layers into both the actor and critic networks of the PPO algorithm. This integration allows for efficient parameter updates of GNNs and enables the state information to be parameterized as a low-dimensional embedding, which is leveraged by the agent to optimize power allocation strategies. Simulation results demonstrate that the proposed method effectively reduces average delay while ensuring user fairness, outperforms baseline methods, and exhibits scalability and generalization capability.

D2D通信强化学习图神经网络功率分配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。