从单智能体到多智能体,系统梳理学习型博弈中的后悔与均衡机制。
Regret, equilibrium, and learning in games: A guided tour
- 基于历史策略的正则化学习,平衡探索与收益
- 在对抗性环境中实现可证明的后悔上界
- 揭示纳什均衡与学习动态稳定的深层联系
本文旨在为学习型博弈研究提供入门指引,涵盖从机器学习、数据科学到经济学等广泛应用。首先分析单个学习者在未知、非平稳甚至对抗环境下的序列决策;随后考察多个相互作用智能体的情形,他们各自追求个体收益最大化,但未必知晓彼此行为或目标。在此框架下,提出一类基于过去行为最优响应的正则化学习策略,通过惩罚项促进探索并避免陷入次优选择。在单智能体场景中,给出对抗性多臂赌博机中正则化学习的基本后悔界;在多智能体场景中,描述零和博弈中遍历均衡收敛结果,类比经典虚构博弈理论,并提出“范式定理”连接战略稳定性(纳什均衡)与动态稳定性(正则化学习吸引点)。重点探讨玩家可用信息,统一分析基于预言机与仅基于收益反馈(赌博机)的方法。目标是呈现该领域近期核心思想的连贯理解,并讨论其对理性研究的启示。
原文摘要 · Abstract (English)
This note aims to serve as an entry point to the literature on learning in games, a topic with significant theoretical appeal and a wide range of applications -- from machine learning and data science to economics and beyond. Our presentation is structured around two complementary viewpoints: We first consider a single agent -- the learner -- engaged in a sequential decision process in an unknown, non-stationary, and possibly adversarial environment. We then examine what happens when the environment is shaped by the decisions of several interacting agents, not necessarily aware of each other's actions or goals, and all seeking to improve their individual rewards. In this general context, we examine a family of regularized learning policies based on best-responding to the past history of play, up to a regularization penalty intended to encourage exploration and prevent over-commitment to suboptimal choices. In the single-agent setting, we present some basic regret bounds for regularized learning in adversarial multi-armed bandits; in the multi-agent setting, we describe an ergodic equilibrium convergence result for zero-sum games in the spirit of classical results on fictitious play, as well as a "folk theorem" linking strategic and dynamic notions of stability -- Nash equilibria and attracting points of regularized learning, respectively. We pay special attention to the information available to the players and, through a unified analysis framework, we study both oracle- and payoff-based (bandit) methods. Our goal is to provide a coherent and comprehensible -- albeit, by necessity, not comprehensive -- account of some recent ideas in the field, and to discuss their implications for the study of rationality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。