针对重尾非平稳老虎机问题,提出新型鲁棒变点检测方法。
Catoni-Style Change Point Detection for Regret Minimization in Non-Stationary Heavy-Tailed Bandits
- 设计基于Catoni估计的重尾分布变点检测策略
- 新算法在合成与真实数据上实现低后悔率
- 适合金融、通信等重尾场景的在线决策应用
近年来,随机非平稳老虎机中的后悔最小化受到广泛关注,可建模从广告到推荐系统等实际问题。现有研究多假设奖励服从伯努利或次高斯分布,但在金融和电信等领域,重尾分布更常见。本文研究重尾分段平稳老虎机问题,其基本假设为:奖励的绝对中心矩最大阶数1+ε(ε∈(0,1])有界于常数v<+∞。考虑最流行的非平稳设置——分段平稳,即奖励分布均值在未知时间点发生改变。我们提出一种新颖的Catoni风格变点检测策略,基于序列估计理论最新进展,具有独立研究价值。进而提出Robust-CPD-UCB算法,结合该检测策略与乐观性老虎机算法,并给出其后悔上界及任意策略的最小可达到后悔的不可能性结果。最后通过合成与真实数据集上的数值实验验证了方法的有效性。
原文摘要 · Abstract (English)
Regret minimization in stochastic non-stationary bandits gained popularity over the last decade, as it can model a broad class of real-world problems, from advertising to recommendation systems. Existing literature relies on various assumptions about the reward-generating process, such as Bernoulli or subgaussian rewards. However, in settings such as finance and telecommunications, heavy-tailed distributions naturally arise. In this work, we tackle the heavy-tailed piecewise-stationary bandit problem. Heavy-tailed bandits, introduced by Bubeck et al., 2013, operate on the minimal assumption that the finite absolute centered moments of maximum order $1+ε$ are uniformly bounded by a constant $v<+\infty$, for some $ε\in (0,1]$. We focus on the most popular non-stationary bandit setting, i.e., the piecewise-stationary setting, in which the mean of reward-generating distributions may change at unknown time steps. We provide a novel Catoni-style change-point detection strategy tailored for heavy-tailed distributions that relies on recent advancements in the theory of sequential estimation, which is of independent interest. We introduce Robust-CPD-UCB, which combines this change-point detection strategy with optimistic algorithms for bandits, providing its regret upper bound and an impossibility result on the minimum attainable regret for any policy. Finally, we validate our approach through numerical experiments on synthetic and real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。