用群体测试降低组合老虎机的计算开销,性能不降。
Combinatorial Multi-armed Bandits: Arm Selection via Group Testing
- 结合群体测试与量化汤普森采样,替代昂贵的最优超臂选择器。
- 在奖励函数满足可分性时,达到与顶尖算法相当的错误率增长速度。
- 特别适合大规模基臂场景,计算复杂度从指数级降至对数级。
本文研究带有半监督反馈和超臂大小基数约束的组合多臂赌博机问题。现有算法通常包含两个关键步骤:(1) 逐次估计基臂参数的参数估计过程;(2) 基于这些参数选择最优子集的超臂选择策略。最先进的算法依赖一个理想化的精确预言机,该预言机在每个时刻需评估数量随臂数线性或指数增长的评分函数,当基臂数量大时计算成本极高。本文提出一种更现实的替代方案:利用群体测试进行超臂选择,同时采用量化汤普森采样完成参数估计。在奖励函数满足一般可分性假设下,所提算法将超臂选择预言机的复杂度降低为基臂数的对数级别,同时保持与使用精确预言机的最先进算法相同的后悔率阶。相比基于预言机的方法,该方法至少实现计算复杂度的指数级降低。
原文摘要 · Abstract (English)
This paper considers the problem of combinatorial multi-armed bandits with semi-bandit feedback and a cardinality constraint on the super-arm size. Existing algorithms for solving this problem typically involve two key sub-routines: (1) a parameter estimation routine that sequentially estimates a set of base-arm parameters, and (2) a super-arm selection policy for selecting a subset of base arms deemed optimal based on these parameters. State-of-the-art algorithms assume access to an exact oracle for super-arm selection with unbounded computational power. At each instance, this oracle evaluates a list of score functions, the number of which grows as low as linearly and as high as exponentially with the number of arms. This can be prohibitive in the regime of a large number of arms. This paper introduces a novel realistic alternative to the perfect oracle. This algorithm uses a combination of group-testing for selecting the super arms and quantized Thompson sampling for parameter estimation. Under a general separability assumption on the reward function, the proposed algorithm reduces the complexity of the super-arm-selection oracle to be logarithmic in the number of base arms while achieving the same regret order as the state-of-the-art algorithms that use exact oracles. This translates to at least an exponential reduction in complexity compared to the oracle-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。