为每个用户定制约束的动态资源分配方案，实现高效公平的网络调度。
Restless Bandits with Individual Penalty Constraints: Near-Optimal Indices and Deep Reinforcement Learning
- 基于惩罚约束设计可离线计算的索引策略，无需在线调整。
- 理论证明该策略渐近最优且满足所有个体约束条件。
- 适用于无线网络等需兼顾公平与效率的实时系统。
本文研究在个体惩罚约束下的扰动多臂赌博机（RMAB）框架，以应对动态无线网络环境中的资源分配挑战。不同于传统模型，本模型允许每个用户（臂）具有不同且严格的性能约束，如能量限制、激活次数上限或信息年龄下限，从而体现公平性与效率等多种目标。为此提出新的惩罚最优惠特尔（POW）索引策略：其索引仅依赖于用户自身的转移核与惩罚约束，不随系统整体特征（如用户数量、可用资源量）变化，支持离线计算且无需在线适应。理论上证明了该策略在满足全部个体惩罚约束的前提下渐近最优。同时引入深度强化学习算法，可在线高效学习POW索引。仿真结果表明，该策略不仅接近最优，且显著优于现有方法。
原文摘要 · Abstract (English)
This paper investigates the Restless Multi-Armed Bandit (RMAB) framework under individual penalty constraints to address resource allocation challenges in dynamic wireless networked environments. Unlike conventional RMAB models, our model allows each user (arm) to have distinct and stringent performance constraints, such as energy limits, activation limits, or age of information minimums, enabling the capture of diverse objectives including fairness and efficiency. To find the optimal resource allocation policy, we propose a new Penalty-Optimal Whittle (POW) index policy. The POW index of an user only depends on the user's transition kernel and penalty constraints, and remains invariable to system-wide features such as the number of users present and the amount of resource available. This makes it computationally tractable to calculate the POW indices offline without any need for online adaptation. Moreover, we theoretically prove that the POW index policy is asymptotically optimal while satisfying all individual penalty constraints. We also introduce a deep reinforcement learning algorithm to efficiently learn the POW index on the fly. Simulation results across various applications and system configurations further demonstrate that the POW index policy not only has near-optimal performance but also significantly outperforms other existing policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。