arXiv:2606.29169cs.GTcs.AI2026-06

提出新算法PED,高效求解多人不完美信息博弈的纳什均衡。

Projected Exploitability Descent for Nash Equilibrium Computation in Multiplayer Imperfect-Information Games

论文配图:Projected Exploitability Descent for Nash Equilibrium Computation in Multiplayer Imperfect-Information Games
图 1 · 摘自论文原文
  • 基于投影子梯度下降最小化可计算的泛化剥削函数代理。
  • 在三玩家德州扑克变体上实现稳定单调收敛,优于传统方法初始阶段。
  • 适合需要长期稳定优化的多人博弈场景,如复杂策略研究。

许多重要博弈涉及多于两人且具有不完美信息。现有计算纳什均衡的方法在这些博弈中或难以扩展,或表现不佳。本文提出一种名为投影剥削下降(Projected Exploitability Descent, PED)的新算法,用于近似求解多人不完美信息博弈中的纳什均衡。该算法通过在可行序列形式策略多面体上运行投影子梯度下降,最小化一个关于多玩家广义剥削函数的代理目标。该目标函数非凸且不可微,但可表示为多个线性函数最大值之和,其子梯度易于计算并可投影至策略可行域。我们在广泛研究的三玩家库恩扑克变体上评估了PED性能。此前无精确算法能扩展至牌堆大小超过4的情况,我们将其与著名的虚构法(FP)和反事实后悔最小化(CFR)算法进行比较。结果表明,尽管FP和CFR在初期迭代中表现更优,但PED在整个运行过程中展现出一致的近单调改进。这启发我们设计混合算法FP-PED:先用FP进行预热,再切换至PED实现长期稳定优化。该过程亦可视为多步算法,将FP作为强初始化步骤用于提升PED性能。

原文摘要 · Abstract (English)

Many important games have more than two players and imperfect information. Existing approaches for computing Nash equilibrium, the central game-theoretic solution concept, in such games either lack scalability or obtain poor performance. In this paper we introduce a new algorithm called projected exploitability descent (PED) for approximating Nash equilibria in multiplayer games of imperfect information. The algorithm works by running projected subgradient descent minimizing a proxy for the multiplayer generalized exploitability function. The objective is nonconvex and nonsmooth, but can be represented as the sum of the maxima of linear functions, for which a subgradient can easily be computed and projected to the polytope of feasible sequence-form strategies. We explore performance of PED on a generalized version of the well-studied benchmark game three-player Kuhn poker. No prior exact algorithms scale to the version of the game with deck size larger than 4, and we compare performance to the popular algorithms of fictitious play (FP) and counterfactual regret minimization (CFR). We find that PED obtains a consistent near-monotonic improvement throughout all runs, though both FP and CFR perform significantly better in the initial iterations. This inspires a hybrid algorithm FP-PED that runs FP for an initial burn-in period before switching to PED for stable long-run refinement. We can alternatively view this as a multi-step algorithm that runs FP as a pre-processing step to obtain a strong initialization for PED.

博弈论纳什均衡多智能体强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。