提出高效算法解决组合博弈中大规模动作选择的最优后悔问题。
An Efficient Minimax-Optimal Algorithm for Adversarial $m$-Set Bandits
- 利用物品损失向量结构,避免枚举所有动作,实现多项式时间计算。
- 在高概率下达到√(dT log(K/δ))的后悔上界,与最优理论极限一致。
- 适用于需要高效处理大规模动作集的在线学习场景,如推荐系统。
我们研究对抗性组合多选博弈(m-set bandits),每轮学习者从 d 个物品中选择 m 个,仅观测所选物品的总损失。动作空间大小为 K = C(d,m),可能指数级增长。但每个动作的损失由同一 d 维物品损失向量决定。本文提出一种计算高效的算法,无需显式枚举动作集即可利用该结构。对自适应非预测对手,以至少 1−δ 的概率保证:相对于最优固定动作的后悔为 R_T = O(√(dT log(K/δ)))。该界与 Zimmert 与 Lattimore 提出的 EXP3-KW 算法最优高概率上界一致,后者直接实现需指数空间。本算法通过仅用 d 个参数表示采样分布,在多项式时间内运行。因此解决了 Maiti 等人提出的开放问题。我们进一步给出匹配的高概率下界:对任意足够小的 δ,存在确定性自适应非预测对手,使得以至少 δ 概率有 R_T = Ω(√(dT log(K/δ)))。故该速率在常数范围内为极小极大最优。特别地,当 m=1 时,证明了普通 K-臂博弈中 log K 项对自适应非预测对手不可避免,填补了 Gerchinovitz 与 Lattimore 未解的 √log K 间隙。
原文摘要 · Abstract (English)
We study adversarial combinatorial bandits with $m$-set actions, where at each round the learner selects $m$ out of $d$ items and observes only the aggregate loss of the selected items. The resulting action set contains $K=\binom{d}{m}$ elements and can therefore be exponentially large. Nevertheless, the loss of every action is determined by the same $d$-dimensional vector of item losses. We propose a computationally efficient algorithm that exploits this structure without explicitly enumerating the action set. Against adaptive non-anticipating adversaries, it guarantees, with probability at least $1-δ$, regret against the best fixed action of \[ R_T = O\left(\sqrt{dT\log(K/δ)}\right). \] This matches the high-probability regret bound of the finite-action EXP3-KW algorithm of Zimmert and Lattimore, whose direct implementation may require exponential space. Our algorithm instead represents each sampling distribution with $d$ parameters and runs in polynomial time without enumerating the action set. Thus, it resolves the open problem posed by Maiti et al. We complement this upper bound with a matching high-probability lower bound. For all sufficiently small $δ$, every randomized policy admits a deterministic adaptive non-anticipating adversary for which, with probability at least $δ$, \[ R_T = Ω\left(\sqrt{dT\log(K/δ)}\right). \] Thus, the rate is minimax optimal up to universal constants in this regime. In particular, setting $m=1$ proves that the $\log K$ for ordinary $K$-armed bandits against adaptive non-anticipating adversaries is unavoidable, closing the remaining $\sqrt{\log K}$ gap between confidence-tuned upper and lower bounds left by Gerchinovitz and Lattimore.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。