已知最优差距时,只需约 log n 次遍历即可高效找出最优选项。
Nearly Tight Bounds for Exploration in Streaming Multi-armed Bandits with Known Optimality Gap
- 利用已知最优差距 Δ_{[2]},设计低内存单臂算法减少遍历次数。
- 证明了在次线性内存下,至少需要 Ω(log n / log log n) 次遍历。
- 提出接近最优的算法,仅需一次臂内存与 O(∑1/Δ²_{[i]]·log n) 次采样。
我们研究了在已知最优差距 Δ_{[2]} 的前提下,多轮流式多臂赌博机(MAB)纯探索问题中的样本-内存-遍历次数权衡。近期工作表明,若未知 Δ_{[2]},则需 Θ(log(1/Δ_{[2]})) 次遍历(忽略 log log 项)才能达到最坏情况下的最优样本复杂度 O(n/Δ²_{[2]}),且仅用单臂内存。然而,对已知 Δ_{[2]} 的多轮算法的理解仍不充分。核心开放问题是:如何在子线性内存下实现复杂度为 O(∑_{i=2}^n 1/Δ²_{[i]}) 的臂采样?本文证明该问题的“正确答案”是 Θ(log n) 次遍历(忽略 log log n 项)。首先,我们建立下界:任何在 o(n/polylog(n)) 内存下,以 O(∑_{i=2}^n 1/Δ²_{[i]} · log n) 次采样找到最优臂的算法,必须进行 Ω(log n / log log n) 次遍历。随后,我们给出一个近乎匹配的算法:假设已知 Δ_{[2]},可在单臂内存下实现总采样量为 O(∑_{i=2}^n 1/Δ²_{[i]} · log n) 的最佳臂识别。
原文摘要 · Abstract (English)
We investigate the sample-memory-pass trade-offs for pure exploration in multi-pass streaming multi-armed bandits (MABs) with the *a priori* knowledge of the optimality gap $Δ_{[2]}$. Here, and throughout, the optimality gap $Δ_{[i]}$ is defined as the mean reward gap between the best and the $i$-th best arms. A recent line of results by Jin, Huang, Tang, and Xiao [ICML'21] and Assadi and Wang [COLT'24] have shown that if there is no known $Δ_{[2]}$, a pass complexity of $Θ(\log(1/Δ_{[2]}))$ (up to $\log\log(1/Δ_{[2]})$ terms) is necessary and sufficient to obtain the *worst-case optimal* sample complexity of $O(n/Δ^{2}_{[2]})$ with a single-arm memory. However, our understanding of multi-pass algorithms with known $Δ_{[2]}$ is still limited. Here, the key open problem is how many passes are required to achieve the complexity, i.e., $O( \sum_{i=2}^{n}1/Δ^2_{[i]})$ arm pulls, with a sublinear memory size. In this work, we show that the ``right answer'' for the question is $Θ(\log{n})$ passes (up to $\log\log{n}$ terms). We first present a lower bound, showing that any algorithm that finds the best arm with slightly sublinear memory -- a memory of $o({n}/{\text{polylog}({n})})$ arms -- and $O(\sum_{i=2}^{n}{1}/{Δ^{2}_{[i]}}\cdot \log{(n)})$ arm pulls has to make $Ω(\frac{\log{n}}{\log\log{n}})$ passes over the stream. We then show a nearly-matching algorithm that assuming the knowledge of $Δ_{[2]}$, finds the best arm with $O( \sum_{i=2}^{n}1/Δ^2_{[i]} \cdot \log{n})$ arm pulls and a *single arm* memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。