为奖励递增的老虎机问题设计了新型贝叶斯算法,可有效降低长期损失。
Thompson Sampling-like Algorithms for Stochastic Rising Bandits
- 基于滑动窗口改进的贝叶斯采样策略,适应奖励随操作上升的特性
- 理论证明在特定条件下算法可实现次线性后悔,且给出后悔下界
- 适用于在线模型选择等需要持续学习的场景,性能优于传统方法
随机上升休息老虎机(SRRB)是一种各选项期望回报随被选次数增加而上升的设置,适用于建模因学习过程导致表现提升的场景(如在线模型选择)。尽管已有基于上置信界的方法,但尚无针对汤普森采样(TS)类算法的研究。本文提出适配滑动窗口的TS算法,并首次完成其在SRRB下的后悔分析,揭示算法成功的关键条件与环境复杂度的决定因素。我们引入一个复杂度指数并建立新的后悔下界。通过合成与真实数据集上的数值实验,验证了所提算法在多种场景下优于现有最优方法。
原文摘要 · Abstract (English)
Stochastic rising rested bandit (SRRB) is a setting where the arms' expected rewards increase as they are pulled. It models scenarios in which the performances of the different options grow as an effect of an underlying learning process (e.g., online model selection). Even if the bandit literature provides specifically crafted algorithms based on upper-confidence bounds for such a setting, no study about Thompson sampling TS-like algorithms has been performed so far. The strong regularity of the expected rewards in the SRRB setting suggests that specific instances may be tackled effectively using adapted and sliding-window TS approaches. This work provides novel regret analyses for such algorithms in SRRBs, highlighting the challenges and providing new technical tools of independent interest. Our results allow us to identify under which assumptions TS-like algorithms succeed in achieving sublinear regret and which properties of the environment govern the complexity of the regret minimization problem when approached with TS. Furthermore, we provide a regret lower bound based on a complexity index we introduce. Finally, we conduct numerical simulations comparing TS-like algorithms with state-of-the-art approaches for SRRBs in synthetic and real-world settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。