无需通信的智能频谱共享,让多个设备自动公平地使用有限频道。
Fair Dynamic Spectrum Access via Fully Decentralized Multi-Agent Reinforcement Learning
- 用强化学习让设备自主选频,不依赖全局信息或彼此通信。
- 在单频道严苛场景下公平性提升89.0%,平均提升48.1%(以Jain指数计)。
- 适合无中心控制的动态无线网络,如物联网或应急通信。
我们研究一个去中心化的无线网络,多个源-目的地对共享有限数量的正交频段。各源节点以去中心化方式自主学习传输策略(特别是频段选择),仅能观测自身传输结果(成功或冲突),无法获知网络规模或其他节点的策略。目标是最大化自身吞吐量的同时实现全网公平。本文提出一种完全去中心化的强化学习方案——公平分摊强化学习(FSRL),结合:(i) 半自适应时间参考的状态增强;(ii) 风险控制与时间差似然的架构设计;(iii) 基于公平性的奖励机制。我们在超过50种网络配置下评估该方法,涵盖不同代理数量、可用频谱量、干扰机存在及自组织场景。仿真表明,在与文献中常见基线算法对比时,FSRL在多源且仅单频段的严苛条件下,公平性(以Jain指数衡量)最高提升89.0%,平均提升48.1%。
原文摘要 · Abstract (English)
We consider a decentralized wireless network with several source-destination pairs sharing a limited number of orthogonal frequency bands. Sources learn to adapt their transmissions (specifically, their band selection strategy) over time, in a decentralized manner, without sharing information with each other. Sources can only observe the outcome of their own transmissions (i.e., success or collision), having no prior knowledge of the network size or of the transmission strategy of other sources. The goal of each source is to maximize their own throughput while striving for network-wide fairness. We propose a novel fully decentralized Reinforcement Learning (RL)-based solution that achieves fairness without coordination. The proposed Fair Share RL (FSRL) solution combines: (i) state augmentation with a semi-adaptive time reference; (ii) an architecture that leverages risk control and time difference likelihood; and (iii) a fairness-driven reward structure. We evaluate FSRL in more than 50 network settings with different number of agents, different amounts of available spectrum, in the presence of jammers, and in an ad-hoc setting. Simulation results suggest that, when we compare FSRL with a common baseline RL algorithm from the literature, FSRL can be up to 89.0% fairer (as measured by Jain's fairness index) in stringent settings with several sources and a single frequency band, and 48.1% fairer on average.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。