arXiv:2509.00179cs.GTcs.LG2025-09

无收益观测下仍可高效对抗对手,适用于马尔可夫博弈。

Playing Markov Games Without Observing Payoffs

  • 基于对手动作序列设计在线学习算法,无需知晓收益。
  • 在对称马尔可夫博弈中,渐近匹配对手回报。
  • 适合研究对抗性学习与信息受限决策的学者。

在多智能体系统中,不确定性下的优化是核心问题。以往研究证明,在重复对称双人矩阵博弈中,只要能观测对手行动,即使不观察收益也能高效竞争。本文提出并形式化了一类新的零和对称马尔可夫博弈,将对称性从矩阵博弈推广到马尔可夫设定。我们证明:即使无法观测收益,只要知晓转移动态并能观察对手的动作序列,玩家仍可对抗拥有完整游戏知识的对手。文中定义了三种对称性概念,并证明在此条件下,学习问题可转化为在线学习实例,使玩家渐近匹配对手收益。算法同时适用于矩阵与马尔可夫博弈,运行时间在游戏规模和轮次上为多项式。本工作拓展了信息劣势下鲁棒学习的适用范围,并深化了在线学习与对抗博弈理论的联系。

原文摘要 · Abstract (English)

Optimization under uncertainty is a fundamental problem in learning and decision-making, particularly in multi-agent systems. Previously, Feldman, Kalai, and Tennenholtz [2010] demonstrated the ability to efficiently compete in repeated symmetric two-player matrix games without observing payoffs, as long as the opponents actions are observed. In this paper, we introduce and formalize a new class of zero-sum symmetric Markov games, which extends the notion of symmetry from matrix games to the Markovian setting. We show that even without observing payoffs, a player who knows the transition dynamics and observes only the opponents sequence of actions can still compete against an adversary who may have complete knowledge of the game. We formalize three distinct notions of symmetry in this setting and show that, under these conditions, the learning problem can be reduced to an instance of online learning, enabling the player to asymptotically match the return of the opponent despite lacking payoff observations. Our algorithms apply to both matrix and Markov games, and run in polynomial time with respect to the size of the game and the number of episodes. Our work broadens the class of games in which robust learning is possible under severe informational disadvantage and deepens the connection between online learning and adversarial game theory.

马尔可夫博弈在线学习对抗学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。