提出自适应分阶段算法,高效解决高维稀疏动作选择的累积损失问题。
Learning to Sparsify Stochastic Linear Bandits

- 用最小二乘法学参数,结合专用子程序选稀疏动作
- 在球形动作集下实现约 $\tilde{\mathcal{O}}(d\sqrt{T})$ 的后悔界
- 适用于推荐系统等需稀疏决策的实际场景
本文研究稀疏随机线性老虎机问题,即决策者从高维动作空间中逐次选择动作,且动作向量非零元素数量受稀疏性约束。核心挑战在于在面对组合结构带来的潜在NP难性时,仍能最小化累积后悔。本文提出一种自适应分阶段探索与利用的算法框架,采用普通最小二乘法进行参数学习,并使用专门子程序进行稀疏动作选择。当动作集为欧几里得球时,可高效计算最优稀疏动作,从而建立 $\tilde{\mathcal{O}}(d\sqrt{T})$ 的后悔上界,其中 $d$ 为动作向量维度,$T$ 为时间跨度。对于一般凸紧动作集(最优稀疏动作不可行),采用贪心子程序:对强凸集,得到 $\tilde{\mathcal{O}}(d\sqrt{T})$ 的 $α$-后悔;对无强凸性的紧集,建立 $\tilde{\mathcal{O}}(d T^{2/3})$ 的 $α$-后悔,其中 $α$ 表示贪心算法的近似比。最后通过大量实验验证算法性能,包括推荐系统应用。
原文摘要 · Abstract (English)
This paper addresses the problem of learning to sparsify stochastic linear bandits, where a decision-maker sequentially selects actions from a high-dimensional space subject to a sparsity constraint on the number of nonzero elements in the action vector. The key challenge lies in minimizing cumulative regret while tackling the potential NP-hardness of finding optimal sparse actions due to the inherent combinatorial structure of the problem. We propose an adaptively phased exploration and exploitation algorithmic framework, utilizing ordinary least squares for parameter learning and specialized subroutines for sparse action selection. When the action set is a Euclidean ball, optimal sparse actions can be efficiently computed, enabling us to establish a $\tilde{\mathcal{O}}(d\sqrt{T})$ regret, where $d$ is the dimension of the action vector and $T$ is the time horizon length. For general convex and compact action sets where finding optimal sparse actions is intractable, we employ a greedy subroutine. For general strongly convex action sets, we derive a $\tilde{\mathcal{O}}(d \sqrt{T})$ $α$-regret; for general compact sets lacking strong convexity, we establish a $\tilde{\mathcal{O}}(d T^{2/3})$ $α$-regret, where $α$ pertains to the approximation ratio of the greedy algorithm. Finally, we validate the performance of our algorithms using extensive experiments including an application to recommendation system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。