FTRL算法在讨价还价游戏中可收敛到近似纳什均衡,突破了以往理论限制。
Last-Iterate Convergence of No-Regret Learning for Equilibria in Bargaining Games
- 使用FTRL算法实现无悔学习,无需零和游戏的特殊修改
- 在独裁者博弈中证明了最后迭代收敛,并给出收敛时间上界
- 适用于多轮讨价还价,能学习出不对称收益的均衡策略
讨价还价游戏是研究经济行为的重要博弈类型,推动了在线学习算法在此类游戏中的研究。本文研究无悔学习算法在讨价还价游戏中收敛至纳什均衡(NE)的条件。尽管近期结果表明,与正则化追随领导者(FTRL)相关的在线算法在多种博弈(包括零和博弈)中可实现最后迭代收敛,但讨价还价游戏缺乏此前建立收敛保证所需的性质,即使在最简单的独裁者博弈(单次“接受或放弃”提议)中亦然。然而,本文证明:在温和假设下,不需为零和博弈进行修正的FTRL算法,在独裁者博弈中仍能实现对近似纳什均衡的最后迭代收敛,并提供收敛时间上界。进一步实验表明,无论初始条件如何,该算法在独裁者博弈及多轮讨价还价游戏中均能收敛至纳什均衡,包括具有非对称收益的均衡。本工作揭示了复杂经济行为(如学习使用威胁、存在多种可能均衡结果)可由简单学习算法产生,且FTRL可在比此前认知更广泛的博弈中收敛至均衡。
原文摘要 · Abstract (English)
Bargaining games, where agents attempt to agree on how to split utility, are an important class of games used to study economic behavior, which motivates a study of online learning algorithms in these games. In this work, we tackle when no-regret learning algorithms converge to Nash equilibria in bargaining games. Recent results have shown that online algorithms related to Follow the Regularized Leader (FTRL) converge to Nash equilibria (NE) in the last iterate in a wide variety of games, including zero-sum games. However, bargaining games do not have the properties used previously to established convergence guarantees, even in the simplest case of the ultimatum game, which features a single take-it-or-leave-it offer. Nonetheless, we establish that FTRL (without the modifications necessary for zero-sum games) achieves last-iterate convergence to an approximate NE in the ultimatum game along with a bound on convergence time under mild assumptions. Further, we provide experimental results to demonstrate that convergence to NE, including NE with asymmetric payoffs, occurs under a broad range of initial conditions, both in the ultimatum game and in bargaining games with multiple rounds. This work demonstrates how complex economic behavior (e.g. learning to use threats and the existence of many possible equilibrium outcomes) can result from using a simple learning algorithm, and that FTRL can converge to equilibria in a more diverse set of games than previously known.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。