arXiv:2505.24193cs.LG2025-05NeurIPS被引 2

新算法在延迟反馈下同时逼近随机与对抗场景的最优误差上限。

Improved Best-of-Both-Worlds Regret for Bandits with Delayed Feedback

  • 设计新算法,分别匹配两类环境下的理论下界。
  • 对抗情形下误差为$\widetilde{O}(\sqrt{KT} + \sqrt{D})$,随机情形下误差更优。
  • 首次在延迟环境下实现双场景最优,适合关注鲁棒性的研究者。

我们研究在对抗性延迟下的多臂赌博机问题,目标是在随机与对抗两种环境中均达到近似最优性能。现有算法在随机设置中仍与已知下界存在显著差距。本文提出一种新算法,可近乎匹配每种情形下的已知下界。在对抗情形下,其后悔值为$\widetilde{O}(\sqrt{KT} + \sqrt{D})$,其中$T$为轮次数,$K$为动作数,$D$为累积延迟,此结果在对数因子内最优。在随机情形下,后悔值为$\sum_{i:Δ_i>0}\left(\log T/Δ_i\right) + \frac{1}{K}\sum Δ_i σ_{max}$,其中$Δ_i$为第$i$个动作的次优差距,$σ_{\max}$为最大缺失观测数。据我们所知,这是首个在延迟环境中同时匹配随机与对抗情形下界的最佳-双世界算法。此外,该随机后悔界是首个在对抗延迟下达到已知下界的,相较此前最优结果,第二项改进了$K$倍。

原文摘要 · Abstract (English)

We study the multi-armed bandit problem with adversarially chosen delays in the Best-of-Both-Worlds (BoBW) framework, which aims to achieve near-optimal performance in both stochastic and adversarial environments. While prior work has made progress toward this goal, existing algorithms suffer from significant gaps to the known lower bounds, especially in the stochastic settings. Our main contribution is a new algorithm that, up to logarithmic factors, matches the known lower bounds in each setting individually. In the adversarial case, our algorithm achieves regret of $\widetilde{O}(\sqrt{KT} + \sqrt{D})$, which is optimal up to logarithmic terms, where $T$ is the number of rounds, $K$ is the number of arms, and $D$ is the cumulative delay. In the stochastic case, we provide a regret bound which scale as $\sum_{i:Δ_i>0}\left(\log T/Δ_i\right) + \frac{1}{K}\sum Δ_i σ_{max}$, where $Δ_i$ is the sub-optimality gap of arm $i$ and $σ_{\max}$ is the maximum number of missing observations. To the best of our knowledge, this is the first BoBW algorithm to simultaneously match the lower bounds in both stochastic and adversarial regimes in delayed environment. Moreover, even beyond the BoBW setting, our stochastic regret bound is the first to match the known lower bound under adversarial delays, improving the second term over the best known result by a factor of $K$.

多臂赌博机延迟反馈后悔分析对抗性延迟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。