未知光滑性下多智能体协同探索，实现无需通信的高效学习。
Coordinating the Unknown Lipschitz Constant in Multiplayer Bandits
- 通过自适应估计光滑性常数并设计同步离散化策略
- 在三种信息结构下均实现最优阶次的累积损失上界
- 适用于去中心化应用，特别适合通信受限场景
针对去中心化应用场景，研究在未知利普希茨常数下的连续动作空间多智能体协作强化学习问题。考虑三种信息结构：(A) 动作不可见但共享奖励，(B) 动作可见且奖励独立，(C) 动作不可见且奖励独立。每种情形下设计并分析了一种算法：估计利普希茨常数，选择联合动作空间的离散化方式，并将协作强化学习方法应用于由此产生的离散问题。学习开始后智能体之间不进行通信，核心挑战在于各方需从各自数据中达成相同的离散化划分。我们证明了：共享奖励与可观测动作可免费提供一致性；在二者缺失时，可通过抖动量化估计值实现一致性，且对主要阶次的遗憾无额外代价。
原文摘要 · Abstract (English)
Motivated by decentralized applications, we study cooperative multi-agent bandits in continuous (Lipschitz) action spaces when the Lipschitz constant is unknown. We consider three information structures: (A)~unobserved actions with common rewards, (B)~observed actions with independent rewards, and (C)~unobserved actions with independent rewards. In each case we design and analyze an algorithm that estimates the Lipschitz constant, chooses a discretization of the joint action space, and applies a cooperative bandit method to the induced discrete problem. Players never communicate once learning starts, so the central difficulty is that they must reach the \emph{same} discretization from their own data. We prove regret guarantees showing that common rewards and observable actions each supply this agreement for free, and that in their absence agreement can still be bought, through a dithered quantization of the estimate, at no cost in the leading order of the regret.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。