arXiv:2510.11691cs.LGcs.GT2025-10被引 2

证明了乐观Hedge在零和博弈中后悔上界最优,常数项也紧。

Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games

  • 通过优化学习率与负项系数,将后悔上界改进为O(√log m log n)
  • 证明现有上界无法进一步改进,社会后悔上下界常数完全匹配
  • 适用于研究博弈学习动态收敛性与后悔分析的学者

在双人零和博弈中,基于乐观Hedge的学习动态在强解耦学习机制中达到了最优后悔上界之一。当学习率选择得当时,社会与个体后悔可被控制在O(log(mn)),其中m、n分别为两玩家的动作数。本文首先改进现有分析,证明在对手动作数已知的强解耦设置下,社会与个体后悔均可优化至O(√log m log n)。该分析将后悔上界转化为关于学习率与负项系数的优化问题,从而精细化主导常数。随后,我们给出依赖于算法的个体后悔下界,表明现有社会后悔上界及新上界均无法再改进。重要的是,社会后悔的上下界在主导项常数上完全一致。最后,基于这些结果,我们提升了基于乐观Hedge的学习动态的最终迭代收敛率与动态后悔界,并提供了匹配的动态后悔下界。

原文摘要 · Abstract (English)

In two-player zero-sum games, the learning dynamic based on optimistic Hedge achieves one of the best-known regret upper bounds among strongly-uncoupled learning dynamics. With an appropriately chosen learning rate, the social and individual regrets can be bounded by $O(\log(mn))$ in terms of the numbers of actions $m$ and $n$ of the two players. This study investigates the optimality of the dependence on $m$ and $n$ in the regret of optimistic Hedge. To this end, we begin by refining existing regret analysis and show that, in the strongly-uncoupled setting where the opponent's number of actions is known, both the social and individual regret bounds can be improved to $O(\sqrt{\log m \log n})$. In this analysis, we express the regret upper bound as an optimization problem with respect to the learning rates and the coefficients of certain negative terms, enabling refined analysis of the leading constants. We then show that the existing social regret bound as well as these new social and individual regret upper bounds cannot be further improved for optimistic Hedge by providing algorithm-dependent individual regret lower bounds. Importantly, these social regret upper and lower bounds match exactly including the constant factor in the leading term. Finally, building on these results, we improve the last-iterate convergence rate and the dynamic regret of a learning dynamic based on optimistic Hedge, and complement these bounds with algorithm-dependent dynamic regret lower bounds that match the improved bounds.

博弈学习后悔分析优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。