提出新算法,解决随机可用性下的带侧信息多臂老虎机问题
An LP-based Sampling Policy for Multi-Armed Bandits with Side-Observations and Stochastic Availability
- 用线性规划动态优化采样策略,兼顾探索与利用
- 理论证明后悔上界受网络结构和激活概率影响
- 适合有依赖关系且资源不稳定的实时决策场景
研究具有侧观测和随机可用性的随机多臂老虎机问题。通过二分图将动作与未知量关联,选择某动作可获得其连接的所有未知量的观测。不同于以往假设所有动作始终可用,本文考虑更现实的动态可用性设定:每轮可行动作集(激活集)随机变化。该框架模拟社交网络中用户虽可提供同伴偏好信息但并非始终在线的情形。为此,提出UCB-LP-A算法,利用线性规划在当前激活动作下优化采样分布,确保有效获取必要观测。理论推导出该策略的后悔上界,揭示网络结构与激活概率的联合影响。数值实验表明,相比忽略侧信息或可用性约束的启发式方法,UCB-LP-A显著更优。
原文摘要 · Abstract (English)
We study the stochastic multi-armed bandit (MAB) problem where an underlying network structure enables side-observations across related actions. We use a bipartite graph to link actions to a set of unknowns, such that selecting an action reveals observations for all the unknowns it is connected to. While previous works rely on the assumption that all actions are permanently accessible, we investigate the more practical setting of stochastic availability, where the set of feasible actions (the "activation set") varies dynamically in each round. This framework models real-world systems with both structural dependencies and volatility, such as social networks where users provide side-information about their peers' preferences, yet are not always online to be queried. To address this challenge, we propose UCB-LP-A, a novel policy that leverages a Linear Programming (LP) approach to optimize exploration-exploitation trade-offs under stochastic availability. Unlike standard network bandit algorithms that assume constant access, UCB-LP-A computes an optimal sampling distribution over the realizable activation sets, ensuring that the necessary observations are gathered using only the currently active arms. We derive a theoretical upper bound on the regret of our policy, characterizing the impact of both the network structure and the activation probabilities. Finally, we demonstrate through numerical simulations that UCB-LP-A significantly outperforms existing heuristics that ignore either the side-information or the availability constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。