改进重尾线性Bandit的后悔界,更优地处理高维和非高斯奖励。
Improved Regret Bounds for Linear Bandits with Heavy-Tailed Rewards
- 设计基于实验设计的消除算法,降低对维度d的依赖。
- 新上界为d^(1+3ε)/2(1+ε) * T^(1/(1+ε)),优于之前结果。
- 适用于高维、无限维及特定几何结构,如Matérn核场景。
我们研究具有重尾奖励的随机线性Bandit问题,其中奖励具有有限的(1+ε)阶绝对中心矩,其上界为υ,ε∈(0,1]。相比已有工作,本文同时改进了最小最大后悔的上下界。当υ=O(1)时,此前最优上界为~O(d T^(1/(1+ε)))。虽然存在相同量级的下界,但其构造依赖于υ=O(d),若将其调整至υ=O(1)情形,仅能得Ω(d^(ε/(1+ε)) T^(1/(1+ε)))下界,该结果与多臂Bandit一致且对线性情形一般较松,尤其在ε=1(有限方差)时比最优率低√d。本文提出一种新的基于实验设计的消除算法,实现~O(d^(1+3ε)/(2(1+ε)) T^(1/(1+ε)))的后悔上界,对所有ε∈(0,1)均改善了d的依赖关系,并在ε=1时恢复已知最优结果。同时建立了Ω(d^(2ε/(1+ε)) T^(1/(1+ε)))的下界,严格优于多臂带问题的率,凸显重尾线性带问题的困难性。对于有限动作集,也推导出相应的改进上下界。此外,给出依赖于动作集结构的上界,表明在某些几何结构(如lp-范数球,p≤1+ε)下可进一步减少对d的依赖;并通过核技巧处理无穷维情形,首次建立针对Matérn核的亚线性后悔界,适用于所有ε∈(0,1]。
原文摘要 · Abstract (English)
We study stochastic linear bandits with heavy-tailed rewards, where the rewards have a finite $(1+ε)$-absolute central moment bounded by $\upsilon$ for some $ε\in (0,1]$. We improve both upper and lower bounds on the minimax regret compared to prior work. When $\upsilon = \mathcal{O}(1)$, the best prior known regret upper bound is $\tilde{\mathcal{O}}(d T^{\frac{1}{1+ε}})$. While a lower with the same scaling has been given, it relies on a construction using $\upsilon = \mathcal{O}(d)$, and adapting the construction to the bounded-moment regime with $\upsilon = \mathcal{O}(1)$ yields only a $Ω(d^{\fracε{1+ε}} T^{\frac{1}{1+ε}})$ lower bound. This matches the known rate for multi-armed bandits and is generally loose for linear bandits, in particular being $\sqrt{d}$ below the optimal rate in the finite-variance case ($ε= 1$). We propose a new elimination-based algorithm guided by experimental design, which achieves regret $\tilde{\mathcal{O}}(d^{\frac{1+3ε}{2(1+ε)}} T^{\frac{1}{1+ε}})$, thus improving the dependence on $d$ for all $ε\in (0,1)$ and recovering a known optimal result for $ε= 1$. We also establish a lower bound of $Ω(d^{\frac{2ε}{1+ε}} T^{\frac{1}{1+ε}})$, which strictly improves upon the multi-armed bandit rate and highlights the hardness of heavy-tailed linear bandit problems. For finite action sets, we derive similarly improved upper and lower bounds for regret. Finally, we provide action set dependent regret upper bounds showing that for some geometries, such as $l_p$-norm balls for $p \le 1 + ε$, we can further reduce the dependence on $d$, and we can handle infinite-dimensional settings via the kernel trick, in particular establishing new regret bounds for the Matérn kernel that are the first to be sublinear for all $ε\in (0, 1]$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。