提出新模型解决稀疏图上的多智能体强化学习难题
Learning Mean Field Control on Sparse Graphs
- 基于局部弱收敛构建稀疏图均场控制模型
- 在幂律网络（指数>2）上实现优于现有方法的性能
- 适合处理真实世界中稀疏复杂网络的多智能体问题
大规模智能体网络在现实应用与自然中广泛存在，因其计算与理论复杂性，给多智能体强化学习（MARL）带来挑战。尽管图函数均场博弈及其扩展适用于稠密或中等稀疏网络，但对更现实的稀疏图仍缺乏有效解决方案。为此，我们提出一种受局部弱收敛启发的新均场控制模型，可处理幂律网络（指数大于2）等稀疏图。结合理论分析，设计了适用于一阶矩有限图序列的可扩展学习算法。在合成与真实网络上的多组实验表明，本方法在多种网络结构中均优于基于Lp图函数和图指数（graphexes）的均场算法，展现出对当前难以解决的稀疏图MARL问题的有效应对能力。
原文摘要 · Abstract (English)
Large agent networks are abundant in applications and nature and pose difficult challenges in the field of multi-agent reinforcement learning (MARL) due to their computational and theoretical complexity. While graphon mean field games and their extensions provide efficient learning algorithms for dense and moderately sparse agent networks, the case of realistic sparser graphs remains largely unsolved. Thus, we propose a novel mean field control model inspired by local weak convergence to include sparse graphs such as power law networks with coefficients above two. Besides a theoretical analysis, we design scalable learning algorithms which apply to the challenging class of graph sequences with finite first moment. We compare our model and algorithms for various examples on synthetic and real world networks with mean field algorithms based on Lp graphons and graphexes. As it turns out, our approach outperforms existing methods in many examples and on various networks due to the special design aiming at an important, but so far hard to solve class of MARL problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。