arXiv:2409.06329stat.MLcs.LG2024-09被引 1

改进的元贝叶斯算法,可自适应线性上下文老虎机任务

Modified Meta-Thompson Sampling for Linear Bandits and Its Bayes Regret Analysis

  • 基于元学习思想,动态调整先验分布以优化探索策略
  • 理论证明其贝叶斯后悔上界为 O((m+log m)√(n log n))
  • 在未见任务上表现良好,适合需要快速适应的新场景

元学习的核心能力是学会如何学习,能够跨任务自适应调整学习策略。近期研究提出元汤普森采样(Meta-TS),通过与来自元先验的老虎机实例交互,元学习未知先验分布。然而,其分析仅限于高斯老虎机。上下文多臂老虎机是高斯老虎机的扩展,要求智能体利用上下文向量预测最优动作,平衡探索与利用以最小化累积后悔。本文提出改进的元汤普森采样线性版本(Meta-TSLB),用于线性上下文老虎机。我们理论分析了 Meta-TSLB,推导出其贝叶斯后悔上界为 $ O((m+ ext{log}(m)) ext{sqrt}(n ext{log}(n))) $,其中 $ m $ 为老虎机实例数,$ n $ 为汤普森采样的轮次数。此外,本工作补充了对线性上下文老虎机中元汤普森采样的分析。实验评估了 Meta-TSLB 在不同设置下的性能,并分析其泛化能力,展示其在未见实例上的适应潜力。

原文摘要 · Abstract (English)

Meta-learning is characterized by its ability to learn how to learn, enabling the adaptation of learning strategies across different tasks. Recent research introduced the Meta-Thompson Sampling (Meta-TS), which meta-learns an unknown prior distribution sampled from a meta-prior by interacting with bandit instances drawn from it. However, its analysis was limited to Gaussian bandit. The contextual multi-armed bandit framework is an extension of the Gaussian Bandit, which challenges agent to utilize context vectors to predict the most valuable arms, optimally balancing exploration and exploitation to minimize regret over time. This paper introduces Meta-TSLB algorithm, a modified Meta-TS for linear contextual bandits. We theoretically analyze Meta-TSLB and derive an $ O((m+\log(m))\sqrt{n\log(n)})$ bound on its Bayes regret, in which $m$ represents the number of bandit instances, and $n$ the number of rounds of Thompson Sampling. Additionally, our work complements the analysis of Meta-TS for linear contextual bandits. The performance of Meta-TSLB is evaluated experimentally under different settings, and we experimente and analyze the generalization capability of Meta-TSLB, showcasing its potential to adapt to unseen instances.

元学习贝叶斯优化上下文老虎机

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。