用Transformer改进强化学习，解决通信网络中的长期依赖与部分可观测问题
Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networks
- 引入Transformer自注意力机制增强RL模型建模长程依赖能力
- 在资源分配、计算卸载等场景中提升决策效率与精度
- 适合对智能通信系统优化感兴趣的科研与工程人员
强化学习（RL）长期以来是解决通信网络各类问题的强大工具。然而，传统RL模型仍面临诸多限制：不仅需要大量与环境交互，且在建模长期关系和处理部分可观测性方面能力有限。近年来，Transformer模型展现出增强RL的能力，其自注意力机制可高效建模长距离依赖与全局相关性，加速训练过程，并支持异构数据模态。本文系统综述了基于Transformer的强化学习算法及其在通信网络中的应用。具体包括RL与Transformer架构的数学基础，以及在资源分配、计算卸载、路由、轨迹控制和网络安全等关键问题上的研究洞察。最后，讨论了当前挑战、开放问题及未来方向，如语义通信与网络优化中增强型深度强化学习算法的应用。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) has long been a powerful solution to various problems in communication networks. However, traditional RL models still face with several limitations. Not only do they rely on large numbers of interactions with the environment, but they are also limited in terms of modeling long-term relationships and tackling partial observability. In recent years, the Transformer model has demonstrated the ability to enhance RL models, allowing them to overcome these issues. Particularly, the self-attention mechanism within the Transformer enables efficient modeling of long-range dependencies and global correlations, as well as accelerates training processes and handles heterogeneous data modalities. In this paper, we present a comprehensive survey of Transformer-based RL algorithms and their applications in communication networks. Specifically, the paper provides the mathematical background of RL and Transformer architectures, along with insights into key issues such as resource allocation, computation offloading, routing, and trajectory control, and network security. We conclude the paper by discussing challenges, open issues, and notable future research directions, including Transformer-enhanced DRL algorithms for semantic communication and network optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。