研究图结构如何影响智能体合作，发现邻居数和路径长度是关键。
The Role of Network Topology and Opponent Information in Shaping Cooperation in Multi-Agent Reinforcement Learning Systems
- 用图结构建模智能体交互，通过深度强化学习学博弈策略。
- 邻居数量多、路径短时更易出现合作行为。
- 知道对手身份反而抑制合作，适合研究合作机制的学者。
本文研究在多智能体强化学习系统中，图拓扑结构与对手信息对合作行为的影响。每个智能体以图中节点形式存在，其邻居构成可交互的对手池。在双人重复囚徒困境（IPD）场景中，智能体通过深度强化学习学习策略，并获得对手的动作历史或身份信息。实验表明，每个节点的邻居数和平均路径长度是决定合作能否出现的主要因素。此外，伙伴选择虽能促进互惠合作，但提供对手身份信息反而阻碍合作策略的传播。
原文摘要 · Abstract (English)
Several works have investigated the influence of graph topology on cooperation among artificial agents, while the majority of the literature has focused on modelling agents' adaptation through strategy imitation, which relies solely on the cumulative payoffs of others. This paper investigates scenarios in which each agent learns to play the two-player Iterated Prisoner's Dilemma (IPD) using deep reinforcement learning. Each agent is represented as a node in a graph, where its neighbours constitute the pool of opponents with whom it can interact. During each IPD episode, agents are provided with different types of information about their opponent, consisting of action history and opponent identity. Experimental results across different graph topologies show that the number of neighbours per node and the average path length are the main factors affecting the emergence of cooperation. We also show that, while partner selection fosters mutual cooperation by limiting the diversity of the opponent pool, providing agents with the identity of their opponent hinders the proliferation of cooperative strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。