在未知线性约束下,快速找到最优且可行的决策方案。
Learning to Explore with Lagrangians for Bandits under Unknown Linear Constraints
- 用拉格朗日松弛法优化探索效率,突破传统方法瓶颈。
- 提出LATS和LAGEX算法,实现渐近最优采样复杂度。
- 适用于资源、安全或公平性受限的超参调优等场景。
纯探索型多臂赌博机可建模超参调优、用户测试等现实问题,其中决策空间常受安全、资源或公平性等线性约束。本文研究在未知线性约束下的纯探索问题,目标是在给定置信度下,以最快速度识别出$r$-最优且可行的策略。首先,提出对带约束纯探索采样复杂度下界的拉格朗日松弛。其次,利用拉格朗日下界中凸优化的性质,设计了两种计算高效的改进算法:LATS与LAGEX,分别基于Track-and-Stop与Gamified Explorer。同时提出一种自适应停止规则,在追踪下界的同时,每步使用可行集的乐观估计。理论证明:LAGEX达到渐近最优的上界;而LATS在新型依赖约束的常数项下渐近最优。数值实验在多种奖励分布与约束条件下验证了LATS和LAGEX的高效性。
原文摘要 · Abstract (English)
Pure exploration in bandits formalises multiple real-world problems, such as tuning hyper-parameters or conducting user studies to test a set of items, where different safety, resource, and fairness constraints on the decision space naturally appear. We study these problems as pure exploration in multi-armed bandits with unknown linear constraints, where the aim is to identify an $r$-optimal and feasible policy as fast as possible with a given level of confidence. First, we propose a Lagrangian relaxation of the sample complexity lower bound for pure exploration under constraints. Second, we leverage properties of convex optimisation in the Lagrangian lower bound to propose two computationally efficient extensions of Track-and-Stop and Gamified Explorer, namely LATS and LAGEX. Then, we propose a constraint-adaptive stopping rule, and while tracking the lower bound, use optimistic estimate of the feasible set at each step. We show that LAGEX achieves asymptotically optimal sample complexity upper bound, while LATS shows asymptotic optimality up to novel constraint-dependent constants. Finally, we conduct numerical experiments with different reward distributions and constraints that validate efficient performance of LATS and LAGEX.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。