针对周期性变化的推荐问题，新算法自动清理旧数据提升决策精度
Periodic Bootstrap Thompson Sampling For Periodically Non-Stationary Bandit Problems
- 周期性重置信念并加入结构化探索，主动清除过时信息
- 在周期性非平稳环境下，累积损失显著低于传统算法
- 适合动态环境中的在线决策，如广告推荐、资源调度
本文提出周期性自助贝叶斯采样（PBTS），一种专为周期性非平稳多臂赌博机问题设计的改进算法。传统贝叶斯采样会累积所有历史观测，导致在奖励分布周期性变化时后验估计偏移。PBTS通过与已知或推断的周期间隔同步重置信念，并引入结构化的自助探索阶段，在清除过时数据的同时保持不确定性估计。在人工构建的环境中测试，涵盖偏斜与均衡奖励分布、不同自助比例及错位周期间隔，结果表明PBTS在周期性非平稳场景下相较传统TS实现统计显著的累积损失降低。研究还讨论了其在真实场景的应用潜力，指出极端周期错位是主要限制，并建议未来工作包括自适应周期识别。该方法通过记忆重置与自助阶段，为周期性奖励场景下的算法优化提供新路径。
原文摘要 · Abstract (English)
This paper introduces Periodic Bootstrap Thompson Sampling (PBTS), an innovative extension of the classic Thompson Sampling (TS) algorithm tailored for bandit problems with periodic non-stationarity. Conventional TS accumulates all past observations, leading to biased posteriors when reward distributions cycle over time. PBTS overcomes this by synchronizing belief resets with known or inferred period intervals and embedding structured bootstrap exploration phases, effectively purging obsolete data while preserving uncertainty estimates. PBTS is tested in artificially constructed environments, which include skewed and balanced reward distributions, along with different bootstrap proportions and misaligned periodic intervals. Results indicate that PBTS generally achieves statistically significant reductions in cumulative regret against traditional TS in periodic non-stationary environments. Subsequent discussion further articulates the potential of PBTS's real-world deployment. The study mentions limitations like extreme periodic misalignment and proposes future research such as self-adjusting cycle-recognition. With memory reset and bootstrap phase, PBTS introduces a novel approach to optimizing bandit algorithms in periodic reward contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。