解决高优先级资源分配的多臂老虎机问题,适用于大模型与边缘智能场景。
Multiple-play Stochastic Bandits with Prioritized Arm Capacity Sharing
- 基于优先级分配容量的多臂老虎机模型,支持动态资源调度。
- 算法实现近最优后悔上界,理论性能逼近下限。
- 适合研究资源调度、强化学习与边缘计算的工程师和学者。
本文提出一种面向大模型应用、边缘智能等场景的资源分配问题的多臂老虎机变体。模型包含 M 个动作臂和 K 次可选操作,每个臂具有随机容量,每单位容量对应一个奖励函数,每次操作具有优先级权重。当多个操作竞争同一臂的容量时,按优先级权重从高到低分配。证明了实例无关与实例相关的后悔下界分别为 Ω(α₁σ√(KM T)) 与 Ω(α₁σ²(M/Δ) ln T),其中 α₁ 为最大优先级权重,σ 表征奖励尾部特性。在参数已知时,设计算法 exttt{MSB-PRS-OffOpt},以 O(MK³) 复杂度求解最优分配策略。利用该算法作为子程序,进一步设计基于置信区间上界的近似算法,其实例无关与实例相关的后悔上界分别与对应下界匹配至 √(K ln KT) 与 α₁K² 因子内。解决了由优先级资源共享机制引发的非线性组合效用函数优化与学习难题。
原文摘要 · Abstract (English)
This paper proposes a variant of multiple-play stochastic bandits tailored to resource allocation problems arising from LLM applications, edge intelligence, etc. The model is composed of $M$ arms and $K$ plays. Each arm has a stochastic number of capacities, and each unit of capacity is associated with a reward function. Each play is associated with a priority weight. When multiple plays compete for the arm capacity, the arm capacity is allocated in a larger priority weight first manner. Instance independent and instance dependent regret lower bounds of $Ω( α_1 σ\sqrt{KM T} )$ and $Ω(α_1 σ^2 \frac{M}Δ \ln T)$ are proved, where $α_1$ is the largest priority weight and $σ$ characterizes the reward tail. When model parameters are given, we design an algorithm named \texttt{MSB-PRS-OffOpt} to locate the optimal play allocation policy with a computational complexity of $O(MK^3)$. Utilizing \texttt{MSB-PRS-OffOpt} as a subroutine, an approximate upper confidence bound (UCB) based algorithm is designed, which has instance independent and instance dependent regret upper bounds matching the corresponding lower bound up to factors of $ \sqrt{K \ln KT }$ and $α_1 K^2$ respectively. To this end, we address nontrivial technical challenges arising from optimizing and learning under a special nonlinear combinatorial utility function induced by the prioritized resource sharing mechanism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。