用一个统一模型搞定多种多智能体强化学习任务
MARL-GPT: Foundation Model for Multi-Agent Reinforcement Learning
- 基于GPT架构的统一模型，无需针对任务调整
- 在三大环境上表现接近专用模型，最高达90%基准
- 适合想减少重复建模的研究者和工业应用
多智能体强化学习近年在多个挑战性场景中取得进展，但通常需为每项任务设计专用模型。本文提出MARL-GPT，一种基于GPT的统一方法，使单一模型可在多种不同环境中学习并高效执行，涵盖StarCraft Multi-Agent Challenge（SMACv2）、Google Research Football（GRF）和POGEMA。该方法利用离线强化学习，在大规模专家轨迹上训练（SMACv2：400M，GRF：100M，POGEMA：1B），仅使用一个无需任务特化调优的Transformer观察编码器。实验表明，MARL-GPT在所有测试环境中均达到与专用基线相当的性能。结果表明，构建适用于广泛异构多智能体问题的通用型变压器模型是可行的，为迈向类似自然语言建模中ChatGPT、Llama等基础模型的多智能体强化学习基础模型铺平道路。
原文摘要 · Abstract (English)
Recent advances in multi-agent reinforcement learning (MARL) have demonstrated success in numerous challenging domains and environments, but typically require specialized models for each task. In this work, we propose a coherent methodology that makes it possible for a single GPT-based model to learn and perform well across diverse MARL environments and tasks, including StarCraft Multi-Agent Challenge, Google Research Football and POGEMA. Our method, MARL-GPT, applies offline reinforcement learning to train at scale on the expert trajectories (400M for SMACv2, 100M for GRF, and 1B for POGEMA) combined with a single transformer-based observation encoder that requires no task-specific tuning. Experiments show that MARL-GPT achieves competitive performance compared to specialized baselines in all tested environments. Thus, our findings suggest that it is, indeed, possible to build a multi-task transformer-based model for a wide variety of (significantly different) multi-agent problems paving the way to the fundamental MARL model (akin to ChatGPT, Llama, Mistral etc. in natural language modeling).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。