提出新算法在成本约束下优化决策,首次实现对数级性能保证。
Pairwise Elimination with Instance-Dependent Guarantees for Bandits with Cost Subsidy
- 设计配对消除算法,按实例依赖方式筛选低效选项。
- 理论证明成本与质量损失均呈对数增长,优于已有方法。
- 适用于需控成本又保收益的场景,如推荐系统优化。
多臂老虎机(MAB)常用于序列在线决策,其中每项决策的回报为未知随机变量。但在实际中,最小化总决策成本可能比最大化总回报更重要,前提是满足回报约束。例如,目标是使决策回报不低于某一已知‘默认’决策的水平,同时尽可能降低成本。该问题被近期引入多臂老虎机带成本补贴(MAB-CS)框架。本文研究了多种变体,包括以已知参考臂回报为约束或以补贴最优回报为约束的情形。针对前者提出配对消除(PE)算法,并将其推广至补贴最优情形的PE-CS。对PE和PE-CS进行实例依赖分析,揭示两者在成本与质量遗憾上均有阶对数上界,为首个具备此类保证的策略。通过上下界对比,证明PE在所有已知参考臂实例中为阶最优。在MovieLens 25M与Goodreads数据集上的实验表明,PE-CS相比文献基线在性能与可靠性间展现出更优平衡。
原文摘要 · Abstract (English)
Multi-armed bandits (MAB) are commonly used in sequential online decision-making when the reward of each decision is an unknown random variable. In practice however, the typical goal of maximizing total reward may be less important than minimizing the total cost of the decisions taken, subject to a reward constraint. For example, we may seek to make decisions that have at least the reward of a reference ``default'' decision, with as low a cost as possible. This problem was recently introduced in the Multi-Armed Bandits with Cost Subsidy (MAB-CS) framework. MAB-CS is broadly applicable to problem domains where a primary metric (cost) is constrained by a secondary metric (reward), and the rewards are unknown. In our work, we address variants of MAB-CS including ones with reward constrained by the reward of a known reference arm or by the subsidized best reward. We introduce the Pairwise-Elimination (PE) algorithm for the known reference arm variant and generalize PE to PE-CS for the subsidized best reward variant. Our instance-dependent analysis of PE and PE-CS reveals that both algorithms have an order-wise logarithmic upper bound on Cost and Quality Regret, making our policies the first with such a guarantee. Moreover, by comparing our upper and lower bound results we establish that PE is order-optimal for all known reference arm problem instances. Finally, experiments are conducted using the MovieLens 25M and Goodreads datasets for both PE and PE-CS revealing the effectiveness of PE and the superior balance between performance and reliability offered by PE-CS compared to baselines from the literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。