用多智能体强化学习优化5G毫米波网络休眠模式，省电又保质量。
Energy Efficient Sleep Mode Optimization in 5G mmWave Networks via Multi Agent Deep Reinforcement Learning
- 采用多智能体深度强化学习，分布式决策降低通信开销。
- 实测节能达0.60 Mbit/Joule，10%分位吞吐率达8.5 Mbps。
- 适合追求低功耗与高可靠性的5G基站部署场景。
在毫米波（mmWave）网络中，动态睡眠模式优化（SMO）对在严格服务质量（QoS）约束下提升能效（EE）至关重要。现有优化与强化学习方法依赖聚合的静态基站（BS）流量模型，无法捕捉非平稳流量动态，且状态-动作空间过大，限制了实际部署。本文提出一种基于双深度Q网络（DDQN）的多智能体强化学习框架（MARL-DDQN），用于三维城市环境中具有时变、社区化用户设备（UE）移动性的自适应SMO。相比传统单智能体方法，MARL-DDQN实现可扩展的分布式决策，信号开销极小。结合真实的基站功耗模型与波束成形，精确量化能效，以吞吐量定义QoS。该方法自适应调整睡眠策略，在抑制小区间干扰的同时保障吞吐公平性。仿真显示，MARL-DDQN优于当前最优方案（包括All On、迭代式QoS感知负载基策略、MARL-DDPG和MARL-PPO），在动态场景下实现最高0.60 Mbit/Joule能效，10%分位吞吐量达8.5 Mbps，95%时间满足QoS要求。
原文摘要 · Abstract (English)
Dynamic sleep mode optimization (SMO) in millimeter-wave (mmWave) networks is essential for maximizing energy efficiency (EE) under stringent quality-of-service (QoS) constraints. However, existing optimization and reinforcement learning (RL) approaches rely on aggregated, static base station (BS) traffic models that fail to capture non-stationary traffic dynamics and suffer from large state-action spaces, limiting real-world deployment. To address these challenges, this paper proposes a multi-agent deep reinforcement learning (MARL) framework using a Double Deep Q-Network (DDQN), referred to as MARL-DDQN, for adaptive SMO in a 3D urban environment with a time-varying and community-based user equipment (UE) mobility model. Unlike conventional single-agent RL, MARL-DDQN enables scalable, distributed decision-making with minimal signaling overhead. A realistic BS power consumption model and beamforming are integrated to accurately quantify EE, while QoS is defined in terms of throughput. The method adapts SMO policies to maximize EE while mitigating inter-cell interference and ensuring throughput fairness. Simulations show that MARL-DDQN outperforms state-of-the-art strategies, including All On, iterative QoS-aware load-based (IT-QoS-LB), MARL-DDPG, and MARL-PPO, achieving up to 0.60 Mbit/Joule EE, 8.5 Mbps 10th-percentile throughput, and meeting QoS constraints 95% of the time under dynamic scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。