用图强化学习优化云网络动态路由,提升稳定性和拓扑感知能力
Topology-Aware Graph Reinforcement Learning for Dynamic Routing in Cloud Networks
- 引入结构感知状态编码与策略自适应图更新机制
- 在GEANT数据集上实现更高吞吐、更低延迟和更好链路均衡
- 适合研究动态网络路由或图强化学习的学者参考
本文提出一种拓扑感知的图强化学习方法,解决云服务器环境中的路由策略优化问题。该方法通过结构感知状态编码(SASE)模块和策略自适应图更新(PAGU)机制,构建统一的状态表示与结构演化框架,以应对动态拓扑下的决策不稳与结构感知不足问题。SASE模块利用多层图卷积与结构位置嵌入建模节点状态,捕捉通信拓扑中的高阶依赖关系,增强状态表达能力。PAGU模块根据策略变化与奖励反馈动态调整图结构,实现环境适应性更新。在真实世界GEANT拓扑数据集上的实验表明,该模型在吞吐量、延迟控制和链路均衡方面均优于多个基准模型。此外,通过超参数敏感性、图稀疏性扰动及节点特征维度变化等实验,验证了结构建模与图更新对模型稳定性与决策质量的影响。结果证明该方法在复杂动态云网络中具备高效且鲁棒的路由性能。
原文摘要 · Abstract (English)
This paper proposes a topology-aware graph reinforcement learning approach to address the routing policy optimization problem in cloud server environments. The method builds a unified framework for state representation and structural evolution by integrating a Structure-Aware State Encoding (SASE) module and a Policy-Adaptive Graph Update (PAGU) mechanism. It aims to tackle the challenges of decision instability and insufficient structural awareness under dynamic topologies. The SASE module models node states through multi-layer graph convolution and structural positional embeddings, capturing high-order dependencies in the communication topology and enhancing the expressiveness of state representations. The PAGU module adjusts the graph structure based on policy behavior shifts and reward feedback, enabling adaptive structural updates in dynamic environments. Experiments are conducted on the real-world GEANT topology dataset, where the model is systematically evaluated against several representative baselines in terms of throughput, latency control, and link balance. Additional experiments, including hyperparameter sensitivity, graph sparsity perturbation, and node feature dimensionality variation, further explore the impact of structure modeling and graph updates on model stability and decision quality. Results show that the proposed method outperforms existing graph reinforcement learning models across multiple performance metrics, achieving efficient and robust routing in dynamic and complex cloud networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。