arXiv:2508.11931cs.LG2025-08NeurIPS被引 2

提出高效算法解决对抗性上下文多臂问题,实现近最优后悔率。

An Improved Algorithm for Adversarial Linear Contextual Bandits via Reduction

  • 通过减少到鲁棒线性带宽问题,仅需线性优化器
  • 在多项式时间内达到 $\widetilde{\mathcal{O}}(\min\{d^2\sqrt{T}, \sqrt{d^3T\log K}\})$ 后悔
  • 首次在多项式时间实现组合带宽的 $\mathrm{poly}(d)\sqrt{T}$ 后悔

我们提出一种口令高效的近似最优算法,用于具有对抗损失和随机动作集的线性上下文带宽问题,每轮仅需动作集上的线性优化口令。该方法将此设置简化为对模型误设鲁棒的对抗线性带宽问题(固定动作集)。在不依赖上下文分布或无上下文模拟器的情况下,算法实现 $\widetilde{\mathcal{O}}(\min\{d^2\sqrt{T}, \sqrt{d^3T\log K}\})$ 的后悔率,运行时间为 $\mathrm{poly}(d,T)$ 加上 $\mathrm{poly}(d,T)$ 次线性优化口令调用,其中 $d$ 为特征维度,$K$ 为每轮动作上限,$T$ 为轮数。这解决了 Liu 等人(2023)提出的开放问题:能否在多项式时间内独立于动作数量获得 $\mathrm{poly}(d)\sqrt{T}$ 后悔率。对于重要的一类对抗损失与随机动作集的组合带宽问题,本算法是首个在多项式时间内实现 $\mathrm{poly}(d)\sqrt{T}$ 后悔率的方法,据我们所知此前无任何算法能在多项式时间内达到 $o(T)$ 后悔。当存在模拟器时,后悔率可进一步优化至 $\widetilde{\mathcal{O}}(d\sqrt{L^\star})$,其中 $L^\star$ 为最优策略的累积损失。

原文摘要 · Abstract (English)

We present an oracle-efficient, near-optimal algorithm for linear contextual bandits with adversarial losses and stochastic action sets, only requiring a linear optimization oracle for the action sets in each round. Our approach reduces this setting to misspecification-robust adversarial linear bandits with fixed action sets. Without knowledge of the context distribution or access to a context simulator, the algorithm achieves $\widetilde{\mathcal{O}}(\min\{d^2\sqrt{T}, \sqrt{d^3T\log K}\})$ regret and runs in $\mathrm{poly}(d,T)$ time plus $\mathrm{poly}(d,T)$ calls to the linear optimization oracles, where $d$ is the feature dimension, $K$ is an upper bound on the number of actions in each round, and $T$ is number of rounds. This resolves the open question by Liu et al. (2023) on whether one can obtain $\mathrm{poly}(d)\sqrt{T}$ regret in polynomial time independent of the number of actions. For the important class of combinatorial bandits with adversarial losses and stochastic action sets, our algorithm is the first to achieve $\mathrm{poly}(d)\sqrt{T}$ regret in polynomial time, while no prior algorithm achieves even $o(T)$ regret in polynomial time to our knowledge. When a simulator is available, the regret bound can be improved to $\widetilde{\mathcal{O}}(d\sqrt{L^\star})$, where $L^\star$ is the cumulative loss of the best policy.

带宽学习对抗性学习优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。