提出高效核化强化学习算法，用探索分布统一多种方法。
Efficient kernelized bandit algorithms via exploration distributions
- 引入探索分布概念，构建可计算的核化强化学习框架
- 实现 $ ilde{O}(γ_T\ oot\sqrt{T}$) 的最优后悔界，匹配现有方法
- 随机化策略在实践中表现更优，适合追求效率的研究者
我们研究一个定义在紧致动作集 $X \subset \mathbb{R}^d$ 上的核化强化学习问题，其未知奖励函数 $f^*$ 在某个再生核希尔伯特空间（RKHS）中具有有限范数。本文提出一类名为 GP-Generic 的高效核化强化学习算法，基于一种新概念——探索分布。该算法族包含基于上置信界（UCB）的方法作为特例，同时也支持多种随机化算法。通过精心设计探索分布，所提通用算法可实现一系列具体算法，达到 $\tilde{O}(γ_T\sqrt{T})$ 的后悔界，其中 $γ_T$ 反映了 RKHS 的复杂度。该结果与已有基于 UCB 及 Thompson Sampling 算法的理论一致；同时实验表明，随机化策略在实际应用中可能带来更优性能。
原文摘要 · Abstract (English)
We consider a kernelized bandit problem with a compact arm set ${X} \subset \mathbb{R}^d $ and a fixed but unknown reward function $f^*$ with a finite norm in some Reproducing Kernel Hilbert Space (RKHS). We propose a class of computationally efficient kernelized bandit algorithms, which we call GP-Generic, based on a novel concept: exploration distributions. This class of algorithms includes Upper Confidence Bound-based approaches as a special case, but also allows for a variety of randomized algorithms. With careful choice of exploration distribution, our proposed generic algorithm realizes a wide range of concrete algorithms that achieve $\tilde{O}(γ_T\sqrt{T})$ regret bounds, where $γ_T$ characterizes the RKHS complexity. This matches known results for UCB- and Thompson Sampling-based algorithms; we also show that in practice, randomization can yield better practical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。