通过中介机制实现多智能体强化学习中的公平领导权分配
Emergence of Fair Leaders via Mediators in Multi-Agent Reinforcement Learning
- 引入中介角色在斯塔克尔伯格博弈中仅负责选择领导者，控制最小
- 自利智能体在中介引导下表现出更公平的行为，整体回报公平性提升
- 适合关注多智能体系统公平性与角色分配的研究者
斯塔克尔伯格博弈及其均衡在多智能体强化学习中受到越来越多关注。传统斯塔克尔伯格博弈中，领导者先行行动，随后是跟随者。当领导者与跟随者的角色可互换时，占据领导地位可能带来显著优势，如先动优势。此时问题浮现：谁应成为领导者？何时轮换？若领导者选择存在偏倚，将导致不公平结果。这一问题在智能体自利、仅关注自身目标和奖励时更为严重。本文正式定义了领导者选择问题，并揭示其与智能体收益公平性的关系。我们提出一种融合中介的多智能体强化学习框架，以最大化公平性。中介此前用于同步行动场景，具有不同程度的控制力，例如直接执行动作或仅推荐动作。本框架首次在斯塔克尔伯格设定中引入中介，且仅通过最小控制（即领导者选择）实现。实验表明，中介的存在促使自利智能体采取公平行为，显著提升整体回报公平性。
原文摘要 · Abstract (English)
Stackelberg games and their resulting equilibria have received increasing attention in the multi-agent reinforcement learning literature. Each stage of a traditional Stackelberg game involves a leader(s) acting first, followed by the followers. In situations where the roles of leader(s) and followers can be interchanged, the designated role can have considerable advantages, for example, in first-mover advantage settings. Then the question arises: Who should be the leader and when? A bias in the leader selection process can lead to unfair outcomes. This problem is aggravated if the agents are self-interested and care only about their goals and rewards. We formally define this leader selection problem and show its relation to fairness in agents' returns. Furthermore, we propose a multi-agent reinforcement learning framework that maximizes fairness by integrating mediators. Mediators have previously been used in the simultaneous action setting with varying levels of control, such as directly performing agents' actions or just recommending them. Our framework integrates mediators in the Stackelberg setting with minimal control (leader selection). We show that the presence of mediators leads to self-interested agents taking fair actions, resulting in higher overall fairness in agents' returns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。