arXiv:2606.09668cs.LG2026-06

提出新算法,将排队调度的后悔率提升至最优阶

Algorithm for Contextual Queueing Bandits with Rate-Optimal Queue Length Regret

论文配图:Algorithm for Contextual Queueing Bandits with Rate-Optimal Queue Length Regret
图 1 · 摘自论文原文
  • 分三阶段设计:随机探索、混合探索与利用、纯利用
  • 队列长度后悔率达 $\widetilde{\mathcal{O}}(T^{-1/2})$,优于旧方法
  • 理论证明最优性,适合学习调度与强化学习研究者

上下文排队带宽提供了一种在未知上下文依赖服务率下学习调度异构任务的框架。在随机上下文假设下,现有算法的队列长度后悔率为 $\widetilde{\mathcal{O}}(T^{-1/4})$,即在时间 $T$ 内学习者与理想策略的队列长度期望差。本文将该速率改进至 $\widetilde{\mathcal{O}}(T^{-1/2})$。核心思想是:随机探索仅需持续到一个精心选择的截止轮次,而非整个时间跨度。我们提出 CQB-$η$-2 算法,包含三阶段:(i) 纯随机探索以构建初始估计器,(ii) 结合 $η$-随机探索与上置信界(UCB)规则,在维持负漂移的同时继续学习,(iii) 截止后转为纯UCB。证明通过分解截止轮次的队列长度后悔量完成:截止前,负漂移抑制次优选择引起的队列差异;截止后,前两阶段提供的足够随机样本确保UCB决策的离开率差距极小。结合两部分边界,总后悔率可达 $\widetilde{\mathcal{O}}(T^{-1/2})$。进一步证明了 $Ω(T^{-1/2})$ 的极小极大下界。构造两个统计上难以区分的困难实例,并使用队列特异性耦合论证,将测试误差转化为队列长度后悔。上下界共同刻画了对时间 $T$ 的极小极大依赖关系,仅差对数因子。

原文摘要 · Abstract (English)

Contextual queueing bandits provide a framework for learning to schedule heterogeneous jobs under unknown context-dependent service rates. Under stochastic contexts, existing algorithms achieve $\widetilde{\mathcal{O}}(T^{-1/4})$ queue length regret, defined as the expected difference between the learner's and oracle's queue lengths at horizon $T$. In this paper, we improve this rate to $\widetilde{\mathcal{O}}(T^{-1/2})$. The key observation is that random exploration is needed only up to a carefully chosen cutoff round, rather than throughout the entire horizon. We propose CQB-$η$-2, a three-phase algorithm: (i) pure random exploration to construct an initial estimator, (ii) $η$-random exploration combined with a UCB rule to continue learning while maintaining negative drift, and (iii) pure UCB after the exploration cutoff. Our proof decomposes the queue length regret at the cutoff round. Before the cutoff, negative drift suppresses queue length differences caused by suboptimal choices. After the cutoff, the first two phases provide sufficient random exploration samples, ensuring that UCB decisions incur small departure-rate gaps. Combining these two bounds yields queue length regret of order $\widetilde{\mathcal{O}}(T^{-1/2})$. We further prove a minimax lower bound of order $Ω(T^{-1/2})$. The proof constructs two hard instances that are statistically indistinguishable up to the final service decision, and uses a queue-specific coupling argument to convert the resulting testing error into queue length regret. Together, our upper and lower bounds characterize the minimax dependence on the horizon $T$ up to logarithmic factors.

强化学习调度优化后悔分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。