让多个智能体像一个整体一样学习,同时保持高效与独立。
Learning Emergence of Interaction Patterns across Independent RL Agents in Multi-Agent Environments
- 将多智能体视为统一整体,用特殊初始化促进独立学习
- 根据梯度动态建立稀疏连接,显著降低计算开销
- 在导航与交通控制任务中表现优于基线方法
许多现实问题,如无人机编队控制和城市交通管理,天然适合建模为多智能体强化学习(RL)问题。然而,现有方法常因智能体间通信引入而面临可扩展性挑战。为此,本文提出一种全新视角:底向上网络(BUN),不再为每个智能体单独建模,而是将多智能体集体视为统一实体,并采用特定权重初始化策略以促进独立学习。同时,BUN通过梯度信息动态构建智能体间的稀疏连接,在需要协作时实现协调,又严格控制连接数量以节省计算资源。在包括协同导航和交通控制在内的多种合作任务中,大量实验证明,相较于基线方法,BUN在性能上具有明显优势,且计算成本大幅下降。
原文摘要 · Abstract (English)
Many real-world problems, such as controlling swarms of drones and urban traffic, naturally lend themselves to modeling as multi-agent reinforcement learning (RL) problems. However, existing multi-agent RL methods often suffer from scalability challenges, primarily due to the introduction of communication among agents. Consequently, a key challenge lies in adapting the success of deep learning in single-agent RL to the multi-agent setting. In response to this challenge, we propose an approach that fundamentally reimagines multi-agent environments. Unlike conventional methods that model each agent individually with separate networks, our approach, the Bottom Up Network (BUN), adopts a unique perspective. BUN treats the collective of multi-agents as a unified entity while employing a specialized weight initialization strategy that promotes independent learning. Furthermore, we dynamically establish connections among agents using gradient information, enabling coordination when necessary while maintaining these connections as limited and sparse to effectively manage the computational budget. Our extensive empirical evaluations across a variety of cooperative multi-agent scenarios, including tasks such as cooperative navigation and traffic control, consistently demonstrate BUN's superiority over baseline methods with substantially reduced computational costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。