arXiv:2506.15943cs.LG2025-06

多智能体个性化线性老虎机中,协作可显著降低累计损失,最优后悔界被完整揭示。

On the optimal regret of collaborative personalized linear bandits

  • 通过分层贝叶斯建模异质性,设计两阶段协作算法实现最优后悔。
  • 在不同异质程度下,后悔界为 $\tilde{O}(d\sqrt{mn})$ 至 $\tilde{O}(dm\sqrt{n})$。
  • 适用于多智能体协同决策场景,尤其适合异质性适中的任务。

随机线性老虎机是序贯决策的基础模型,智能体选择向量动作并获得带噪声的奖励,其期望值由未知线性函数决定。尽管单智能体情形已深入研究,但现实场景常涉及多个智能体解决异构问题,各自参数不同。独立应用单智能体算法会忽略跨智能体相似性与学习机会。本文研究协作个性化线性老虎机中的最优后悔。我们给出信息论下界,刻画智能体数 $m$、交互轮数 $n$ 与异质性程度 $σ$ 如何联合影响后悔。随后提出一种新两阶段协作算法,达到最优后悔。分析基于分层贝叶斯框架建模异质性,并引入新颖信息论技术。结果完整刻画协作优势:当 $n \in (0, \frac{d}{m σ^2})$ 时,后悔界为 $\tilde{O}(d\sqrt{mn})$;当 $n \in [\frac{d}{m^{2γ} σ^2}, \frac{d}{σ^2}]$ 时,为 $\tilde{O}(dm^{1-γ}\sqrt{n})$;当 $n > \frac{d}{σ^2}$ 时,为 $\tilde{O}(dm\sqrt{n})$,其中 $γ \in [0, 1/2]$ 为绝对常数。而无协作时,最坏后悔界为 $O(dm\sqrt{n})$。

原文摘要 · Abstract (English)

Stochastic linear bandits are a fundamental model for sequential decision making, where an agent selects a vector-valued action and receives a noisy reward with expected value given by an unknown linear function. Although well studied in the single-agent setting, many real-world scenarios involve multiple agents solving heterogeneous bandit problems, each with a different unknown parameter. Applying single agent algorithms independently ignores cross-agent similarity and learning opportunities. This paper investigates the optimal regret achievable in collaborative personalized linear bandits. We provide an information-theoretic lower bound that characterizes how the number of agents, the interaction rounds, and the degree of heterogeneity jointly affect regret. We then propose a new two-stage collaborative algorithm that achieves the optimal regret. Our analysis models heterogeneity via a hierarchical Bayesian framework and introduces a novel information-theoretic technique for bounding regret. Our results offer a complete characterization of when and how collaboration helps with a optimal regret bound $\tilde{O}(d\sqrt{mn})$, $\tilde{O}(dm^{1-γ}\sqrt{n})$, $\tilde{O}(dm\sqrt{n})$ for the number of rounds $n$ in the range of $(0, \frac{d}{m σ^2})$, $[\frac{d}{m^{2γ} σ^2}, \frac{d}{σ^2}]$ and $(\frac{d}{σ^2}, \infty)$ respectively, where $σ$ measures the level of heterogeneity, $m$ is the number of agents, and $γ\in[0, 1/2]$ is an absolute constant. In contrast, agents without collaboration achieve a regret bound $O(dm\sqrt{n})$ at best.

多智能体线性老虎机协作学习后悔界

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。