通过随机采样减少计算量，让多智能体强化学习更高效。
Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning
- 用随机采样部分智能体来近似全局策略，降低复杂度。
- 采样数量每增加一倍，误差下降约1/√k，与总智能体数无关。
- 适合大规模多智能体系统，尤其适用于分布式决策场景。
多智能体强化学习（MARL）的挑战在于联合状态与动作空间随智能体数量呈指数级增长。当需要兼顾全局序列决策与局部交互时，这一问题更加严重。本文提出新算法 SUBSAMPLE-MFQ 和一种去中心化随机策略，针对 n 个智能体的系统设计。对于任意 k ≤ n，该算法可在 k 的多项式时间内学习系统策略。我们证明，随着采样智能体数 k 增加，所学策略收敛到最优策略的误差为 Õ(1/√k)，且该界与智能体总数 n 无关。
原文摘要 · Abstract (English)
Designing efficient algorithms for multi-agent reinforcement learning (MARL) is fundamentally challenging because the size of the joint state and action spaces grows exponentially in the number of agents. These difficulties are exacerbated when balancing sequential global decision-making with local agent interactions. In this work, we propose a new algorithm $\texttt{SUBSAMPLE-MFQ}$ ($\textbf{Subsample}$-$\textbf{M}$ean-$\textbf{F}$ield-$\textbf{Q}$-learning) and a decentralized randomized policy for a system with $n$ agents. For any $k\leq n$, our algorithm learns a policy for the system in time polynomial in $k$. We prove that this learned policy converges to the optimal policy on the order of $\tilde{O}(1/\sqrt{k})$ as the number of subsampled agents $k$ increases. In particular, this bound is independent of the number of agents $n$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。