用Transformer改进强化学习,实现更智能的网络服务链分割。
Transformer-Empowered Actor-Critic Reinforcement Learning for Sequence-Aware Service Function Chain Partitioning
- 引入Transformer自注意力机制建模VNF间依赖关系。
- 在仿真中提升服务接纳率与资源利用率,支持快速推理。
- 适合6G网络中的动态服务编排与大规模部署场景。
在未来的6G网络中,以超高速率、超低时延和广泛连接为特征,虚拟化网络功能(VNFs)的有效管理至关重要。服务功能链(SFCs)作为有序的VNF序列,是提供复杂网络服务的关键。然而,将SFC拆分并部署到不同网络域或基础设施位置时,受限于域间异构性、服务质量(QoS)约束以及网络状态可见性不足,面临巨大挑战。传统优化方法可扩展性差,现有数据驱动方法难以兼顾效率与VNF间依赖建模。为此,我们提出一种基于Transformer的演员-评论家强化学习框架,专用于序列感知的SFC分割。通过自注意力机制,有效建模VNF间的复杂依赖关系,实现协同与并行决策。同时,为提升训练稳定性和收敛性,引入ε-LoPe探索策略与渐近回报归一化。综合仿真结果表明,该方法在长期服务接纳率、资源利用率和可扩展性方面均优于现有最先进方案,并实现快速推理。
原文摘要 · Abstract (English)
In the forthcoming era of 6G networks, characterized by unprecedented data rates, ultra-low latency, and ubiquitous connectivity, effective management of Virtualized Network Functions (VNFs) is essential. VNFs are software-based counterparts of traditional hardware devices that facilitate flexible and scalable service provisioning. Service Function Chains (SFCs), structured as ordered sequences of VNFs, are pivotal in delivering complex network services. Nevertheless, splitting an SFC into multiple segments that are deployed across different network domains or infrastructure locations presents substantial challenges due to the potential heterogeneity of domain characteristic along with quality of service (QoS) constraints and limited visibility of network state. Conventional optimization methods have limited scalability, while existing data-driven approaches struggle to balance efficiency with capturing VNF inter-dependencies in SFCs. To overcome these limitations, we introduce a Transformer-empowered actor-critic framework specifically designed for sequence-aware SFC partitioning. By utilizing the self-attention mechanism, our approach effectively models complex inter-dependencies between VNFs, facilitating coordinated and parallel decision-making processes. Furthermore, to improve training stability and convergence we introduce an $ε$-LoPe exploration strategy as well as Asymptotic Return Normalization. Comprehensive simulation results demonstrate that the proposed methodology outperforms existing state-of-the-art solutions in terms of long-term service acceptance rates, resource utilization, and scalability while achieving fast inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。