用离线数据迁移超参数，让随机老虎机算法更快适应新任务。
Offline-to-online hyperparameter transfer for stochastic bandits
- 从多个任务的离线数据中学习超参数，用于新任务初始化。
- 理论上证明了所需任务数和每任务尝试次数的下界。
- 适用于UCB、LinUCB等经典算法，适合实际在线学习场景。
经典随机老虎机算法依赖超参数控制探索与利用的权衡。调参具有重要实践意义，但常面临信息论上的不可能性。为此，我们考虑一种实用的迁移学习场景：已知若干来自未知任务分布的离线数据。目标是利用这些数据为从同一分布中抽样的新任务设定近优超参数。我们给出了在未知任务上学习近优超参数所需的跨任务（任务数量）和内任务（每任务臂选择次数）样本复杂度上界。该理论适用于多种经典算法，包括调整UCB和LinUCB中的探索参数，以及GP-UCB中的噪声参数。实验表明，从离线问题迁移超参数在随机老虎机在线学习中具有显著效果。
原文摘要 · Abstract (English)
Classic algorithms for stochastic bandits typically use hyperparameters that govern their critical properties such as the trade-off between exploration and exploitation. Tuning these hyperparameters is a problem of great practical significance. However, this is a challenging problem and in certain cases is information theoretically impossible. To address this challenge, we consider a practically relevant transfer learning setting where one has access to offline data collected from several bandit problems (tasks) coming from an unknown distribution over the tasks. Our aim is to use this offline data to set the hyperparameters for a new task drawn from the unknown distribution. We provide bounds on the inter-task (number of tasks) and intra-task (number of arm pulls for each task) sample complexity for learning near-optimal hyperparameters on unseen tasks drawn from the distribution. Our results apply to several classic algorithms, including tuning the exploration parameters in UCB and LinUCB and the noise parameter in GP-UCB. Our experiments indicate the significance and effectiveness of the transfer of hyperparameters from offline problems in online learning with stochastic bandit feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。