提出风险感知的随机老虎机算法，降低决策中的潜在损失风险。
Risk-Aware Decision Making in Restless Bandits: Theory and Algorithms for Planning and Learning
- 引入风险意识机制，用威特指数法解决非平稳与平稳情形下的规划问题。
- 在未知转移概率时，采用汤普森采样，实现亚线性与二次方的遗憾增长。
- 适用于设备更换和病人调度等需规避风险的实际场景，适合决策优化研究者。
在非平稳有限时域与平稳无限时域马尔可夫决策过程下，针对资源受限的多臂老虎机问题，本文首次将传统风险中性目标推广至风险感知框架，建立索引可解性条件并提出基于威特指数的求解方法。当真实转移概率未知时，设计了一种汤普森采样算法，证明其后悔上界在轮次数上亚线性、在臂数上二次增长。通过机器更换与患者调度等数值实验，验证了该方法在降低风险暴露方面的有效性。
原文摘要 · Abstract (English)
In restless bandits, a central agent is tasked with optimally distributing limited resources across several bandits (arms), with each arm being a Markov decision process. In this work, we generalize the traditional restless bandits problem with a risk-neutral objective by incorporating risk-awareness, which is particularly important in various real-world applications especially when the decision maker seeks to mitigate downside risks. We establish indexability conditions for the case of a risk-aware objective and provide a solution based on Whittle index for the first time for the planning problem with finite-horizon non-stationary and for infinite-horizon stationary Markov decision processes. In addition, we address the learning problem when the true transition probabilities are unknown by proposing a Thompson sampling approach and show that it achieves bounded regret that scales sublinearly with the number of episodes and quadratically with the number of arms. The efficacy of our method in reducing risk exposure in restless bandits is illustrated through a set of numerical experiments in the contexts of machine replacement and patient scheduling applications under both planning and learning setups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。