提出高效算法CMOSS,实现近最优后悔值且无对数时间依赖。
Near-Optimal Regret for Efficient Stochastic Combinatorial Semi-Bandits
- 基于最小最大优化思想设计新算法,兼顾效率与性能。
- 在半反馈下达到理论最优后悔上界,消除额外log T项。
- 适合长期决策场景,尤其适用于大规模组合选择问题。
组合多臂老虎机(CMAB)是序贯决策的核心框架,主流方法分为基于UCB和对抗性方法(如FTRL、OMD)。然而,典型UCB方法(如CUCB)存在随时间增长的log T后悔因子,而对抗性方法(如EXP3.M、HYBRID)计算开销大。为解决这一权衡,本文提出随机设定下的组合最小最大最优策略(CMOSS)。CMOSS在半反馈下,当k≤m/2时实现实例无关后悔上界O((log k)√(kmT)),当k>m/2时为O((m−k)√(log k log(m−k)T)),彻底消除log T依赖,并在两种情况下分别匹配Ω(√(kmT))和Ω((m−k)√(log(m/(m−k))T))的已知下界,仅差对数因子。进一步分析表明,该方法可推广至级联反馈场景。合成数据与真实数据实验验证,CMOSS在后悔值和运行效率上均优于基准算法。
原文摘要 · Abstract (English)
The combinatorial multi-armed bandit (CMAB) is a cornerstone of sequential decision-making framework, dominated by two algorithmic families: UCB-based and adversarial methods such as follow the regularized leader (FTRL) and online mirror descent (OMD). However, prominent UCB-based approaches like CUCB suffer from additional regret factor $\log T$ that is detrimental over long horizons, while adversarial methods such as EXP3.M and HYBRID impose significant computational overhead. To resolve this trade-off, we introduce the Combinatorial Minimax Optimal Strategy in the Stochastic setting (CMOSS). CMOSS is a computationally efficient algorithm that achieves an instance-independent regret of $O\big( (\log k)\sqrt{kmT}\big )$ when $k\leq \frac{m}{2}$ and $O\big((m-k)\sqrt{\log k\log(m-k)T}\big )$ when $k>\frac{m}{2}$ under semi-bandit feedback, where $m$ is the number of arms and $k$ is the maximum cardinality of a feasible action. Crucially, this result eliminates the dependency on $\log T$ and matches the established lower bounds of $Ω\big(\sqrt{kmT}\big)$ when $k\leq \frac{m}{2}$ and $Ω\big((m-k)\sqrt{\log (\frac{m}{m-k}) T}\big)$ when $k>\frac{m}{2}$ up to logarithmic terms of $k$ and $m$. We then extend our analysis to show that CMOSS is also applicable to cascading feedback. Experiments on synthetic and real-world datasets validate that CMOSS consistently outperforms benchmark algorithms in both regret and runtime efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。