arXiv:2508.07001cs.NIcs.AI2025-08中稿 · ACM International …被引 5

无需中心化训练,通过设备间共识优化随机接入网络

Consensus-based Decentralized Multi-agent Reinforcement Learning for Random Access Network Optimization

论文配图:Consensus-based Decentralized Multi-agent Reinforcement Learning for Random Access Network Optimization
图 1 · 摘自论文原文
  • 采用基于共识的去中心化强化学习,仅交换本地奖励
  • 理论证明全局收敛性,实验显示显著降低冲突率
  • 适合大规模无线设备协同场景,通信开销极低

随着无线设备日益形成统一智能网络以实现无缝、便捷操作,随机接入(RA)介质访问控制(MAC)设计被视为应对多终端不可预测数据流量的关键方案。然而,设计一种有效减少冲突并保证设备间传输公平性的RA-MAC协议仍具挑战。现有基于集中式训练、分布式执行(CTDE)的多智能体强化学习(MARL)方法虽可优化性能,但依赖中心化训练及大量信息收集,难以在真实场景应用。本文提出完全去中心化的MARL架构,策略学习不依赖中心任务,而是通过设备间的共识机制交换信息。我们在演员-评论家(AC)网络上设计算法,仅交换本地奖励以最小化通信开销。此外,我们提供了该方法全局收敛性的理论证明。数值实验表明,所提MARL算法相比其他基线显著提升随机接入网络性能。

原文摘要 · Abstract (English)

With wireless devices increasingly forming a unified smart network for seamless, user-friendly operations, random access (RA) medium access control (MAC) design is considered a key solution for handling unpredictable data traffic from multiple terminals. However, it remains challenging to design an effective RA-based MAC protocol to minimize collisions and ensure transmission fairness across the devices. While existing multi-agent reinforcement learning (MARL) approaches with centralized training and decentralized execution (CTDE) have been proposed to optimize RA performance, their reliance on centralized training and the significant overhead required for information collection can make real-world applications unrealistic. In this work, we adopt a fully decentralized MARL architecture, where policy learning does not rely on centralized tasks but leverages consensus-based information exchanges across devices. We design our MARL algorithm over an actor-critic (AC) network and propose exchanging only local rewards to minimize communication overhead. Furthermore, we provide a theoretical proof of global convergence for our approach. Numerical experiments show that our proposed MARL algorithm can significantly improve RA network performance compared to other baselines.

去中心化强化学习随机接入多智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。