arXiv:2508.06247cs.LGcs.DS2025-08

提出高效算法CMOSS,实现近最优后悔值且无对数时间依赖。

Near-Optimal Regret for Efficient Stochastic Combinatorial Semi-Bandits

  • 基于最小最大优化思想设计新算法,兼顾效率与性能。
  • 在半反馈下达到理论最优后悔上界,消除额外log T项。
  • 适合长期决策场景,尤其适用于大规模组合选择问题。

组合多臂老虎机(CMAB)是序贯决策的核心框架,主流方法分为基于UCB和对抗性方法(如FTRL、OMD)。然而,典型UCB方法(如CUCB)存在随时间增长的log T后悔因子,而对抗性方法(如EXP3.M、HYBRID)计算开销大。为解决这一权衡,本文提出随机设定下的组合最小最大最优策略(CMOSS)。CMOSS在半反馈下,当k≤m/2时实现实例无关后悔上界O((log k)√(kmT)),当k>m/2时为O((m−k)√(log k log(m−k)T)),彻底消除log T依赖,并在两种情况下分别匹配Ω(√(kmT))和Ω((m−k)√(log(m/(m−k))T))的已知下界,仅差对数因子。进一步分析表明,该方法可推广至级联反馈场景。合成数据与真实数据实验验证,CMOSS在后悔值和运行效率上均优于基准算法。

原文摘要 · Abstract (English)

The combinatorial multi-armed bandit (CMAB) is a cornerstone of sequential decision-making framework, dominated by two algorithmic families: UCB-based and adversarial methods such as follow the regularized leader (FTRL) and online mirror descent (OMD). However, prominent UCB-based approaches like CUCB suffer from additional regret factor $\log T$ that is detrimental over long horizons, while adversarial methods such as EXP3.M and HYBRID impose significant computational overhead. To resolve this trade-off, we introduce the Combinatorial Minimax Optimal Strategy in the Stochastic setting (CMOSS). CMOSS is a computationally efficient algorithm that achieves an instance-independent regret of $O\big( (\log k)\sqrt{kmT}\big )$ when $k\leq \frac{m}{2}$ and $O\big((m-k)\sqrt{\log k\log(m-k)T}\big )$ when $k>\frac{m}{2}$ under semi-bandit feedback, where $m$ is the number of arms and $k$ is the maximum cardinality of a feasible action. Crucially, this result eliminates the dependency on $\log T$ and matches the established lower bounds of $Ω\big(\sqrt{kmT}\big)$ when $k\leq \frac{m}{2}$ and $Ω\big((m-k)\sqrt{\log (\frac{m}{m-k}) T}\big)$ when $k>\frac{m}{2}$ up to logarithmic terms of $k$ and $m$. We then extend our analysis to show that CMOSS is also applicable to cascading feedback. Experiments on synthetic and real-world datasets validate that CMOSS consistently outperforms benchmark algorithms in both regret and runtime efficiency.

强化学习多臂老虎机后悔分析组合优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。