让智能体通过共情动态平衡利他与自保,避免被剥削。
Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games
- 基于共情评估同伴友好度,动态分配奖励作为赠礼。
- 在扩展的混合动机游戏中实现高效协作且公平不被占便宜。
- 适合研究多智能体合作与对抗平衡的学者使用。
现实中的多智能体场景常涉及混合动机,要求智能体既具备利他能力又可防范潜在剥削。现有方法难以兼顾两者。本文提出LASE(Learning to balance Altruism and Self-interest based on Empathy),一种分布式多智能体强化学习算法,通过赠礼促进利他合作,同时避免被其他智能体剥削。LASE将部分奖励动态分配给合作者,分配比例根据社会关系(social relationship)调整——该指标通过反事实推理估算同伴行为的友好程度。具体而言,社会关系通过比较当前联合动作下的$Q$-函数与忽略该同伴动作的反事实基线,结合视角转换模块推断其行动分布来衡量。在空间和时间上扩展的混合动机游戏中进行了全面实验,结果表明LASE能有效促进群体协作而不牺牲公平性,并能适应不同类型的交互伙伴。
原文摘要 · Abstract (English)
Real-world multi-agent scenarios often involve mixed motives, demanding altruistic agents capable of self-protection against potential exploitation. However, existing approaches often struggle to achieve both objectives. In this paper, based on that empathic responses are modulated by inferred social relationships between agents, we propose LASE Learning to balance Altruism and Self-interest based on Empathy), a distributed multi-agent reinforcement learning algorithm that fosters altruistic cooperation through gifting while avoiding exploitation by other agents in mixed-motive games. LASE allocates a portion of its rewards to co-players as gifts, with this allocation adapting dynamically based on the social relationship -- a metric evaluating the friendliness of co-players estimated by counterfactual reasoning. In particular, social relationship measures each co-player by comparing the estimated $Q$-function of current joint action to a counterfactual baseline which marginalizes the co-player's action, with its action distribution inferred by a perspective-taking module. Comprehensive experiments are performed in spatially and temporally extended mixed-motive games, demonstrating LASE's ability to promote group collaboration without compromising fairness and its capacity to adapt policies to various types of interactive co-players.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。