在未知分布情况下,用最少尝试次数识别分组结构,适合带反馈的在线聚类任务。
A General Framework for Clustering and Distribution Matching with Bandit Feedback
- 基于追踪停止与Frank-Wolfe算法设计高效在线学习策略
- 理论证明平均尝试次数逼近最优下界,误差率可控在δ内
- 可解决配对、奇数臂识别等经典问题,适用于资源受限场景
我们提出一个通用框架,用于在贝叶斯反馈下进行聚类与分布匹配。考虑一个含K个动作的多臂老虎机模型,其中部分动作被划分为M个组,每组内各动作的随机变量服从同一有限字母表上的分布。每个时刻决策者选择一个动作并观测其结果,后续选择依赖于历史记录。决策者对动作分布及分组结构一无所知。目标是设计在线算法,在平均尝试次数最少且错误概率不超过预设值δ的前提下,学习出原始分组结构。多个已有问题,如寻找M对动作、奇数臂识别、K个动作的N元聚类,均属于该框架。我们推导了任意在线算法在错误概率不超过δ时,平均尝试次数的非渐近下界。进一步,我们提出一种基于追踪停止与Frank-Wolfe算法的计算高效算法,并证明其平均尝试次数渐近逼近该下界。精细化分析揭示了算法平均尝试次数向理论极限收敛的速度新界限,当δ趋于0时具有明确收敛速率。
原文摘要 · Abstract (English)
We develop a general framework for clustering and distribution matching problems with bandit feedback. We consider a $K$-armed bandit model where some subset of $K$ arms is partitioned into $M$ groups. Within each group, the random variable associated to each arm follows the same distribution on a finite alphabet. At each time step, the decision maker pulls an arm and observes its outcome from the random variable associated to that arm. Subsequent arm pulls depend on the history of arm pulls and their outcomes. The decision maker has no knowledge of the distributions of the arms or the underlying partitions. The task is to devise an online algorithm to learn the underlying partition of arms with the least number of arm pulls on average and with an error probability not exceeding a pre-determined value~$δ$. Several existing problems fall under our general framework, including finding $M$ pairs of arms, odd arm identification, and $N$-ary clustering of $K$ arms belong to our general framework. We derive a non-asymptotic lower bound on the average number of arm pulls for any online algorithm with an error probability not exceeding $δ$. Furthermore, we develop a computationally-efficient online algorithm based on the Track-and-Stop method and Frank--Wolfe algorithm, and show that the average number of arm pulls of our algorithm asymptotically matches that of the lower bound. Our refined analysis also uncovers a novel bound on the speed at which the average number of arm pulls of our algorithm converges to the fundamental limit as $δ$ vanishes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。