用深度强化学习同步优化路径与交换节点，加速软件定义网络的流表下发。
A New Segment Routing method with Swap Node Selection Strategy Based on Deep Reinforcement Learning for Software Defined Network
- 设计基于DRL的智能路由与分段策略联合优化方法。
- 相比传统方法，流表下发时间降低23.7%，吞吐量提升18.5%。
- 适合需要低延迟、高吞吐的SDN场景，如数据中心网络。
现有分段路由（SR）方法需先确定路径，再通过路径分段选择交换节点形成分段路由路径（SRP），路径变更时需重新分段，且未考虑流表下发时间，无法最大化下发速度。为此，本文构建了可同时优化路由与路径分段策略的模型，以选择合适的交换节点来减少流表下发时间。设计了一种基于深度强化学习的智能分段路由算法（DRL-SR）求解该模型：首先将包含多类QoS指标、流表下发时间开销和SR标签栈深度的流量矩阵作为强化学习智能体的状态空间；其次设计动作选择策略与奖励函数，智能体在选择下一跳节点时考虑路由，同时判断新节点是否作为交换节点，并结合控制器向交换节点下发流表的时间成本设计奖励机制。实验结果表明，相较于现有方法，所提出的分段路由优化模型与智能算法（DRL-SR）在完成分段路由建立任务时显著降低时间开销，同时优化了吞吐量、延迟和丢包率等性能指标。
原文摘要 · Abstract (English)
The existing segment routing (SR) methods need to determine the routing first and then use path segmentation approaches to select swap nodes to form a segment routing path (SRP). They require re-segmentation of the path when the routing changes. Furthermore, they do not consider the flow table issuance time, which cannot maximize the speed of issuance flow table. To address these issues, this paper establishes an optimization model that can simultaneously form routing strategies and path segmentation strategies for selecting the appropriate swap nodes to reduce flow table issuance time. It also designs an intelligent segment routing algorithm based on deep reinforcement learning (DRL-SR) to solve the proposed model. First, a traffic matrix is designed as the state space for the deep reinforcement learning agent; this matrix includes multiple QoS performance indicators, flow table issuance time overhead and SR label stack depth. Second, the action selection strategy and corresponding reward function are designed, where the agent selects the next node considering the routing; in addition, the action selection strategy whether the newly added node is selected as the swap node and the corresponding reward function are designed considering the time cost factor for the controller to issue the flow table to the swap node. Finally, a series of experiments and their results show that, compared with the existing methods, the designed segmented route optimization model and the intelligent solution algorithm (DRL-SR) can reduce the time overhead required to complete the segmented route establishment task while optimizing performance metrics such as throughput, delays and packet losses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。