用熵风险度量高效求解马尔可夫决策中的风险敏感规划
Efficient Risk-sensitive Planning via Entropic Risk Measures
- 通过结构分析与光滑性性质，高效计算熵风险度量的最优策略前沿
- 在多种决策场景中性能优异，逼近目标风险指标的最优解
- 无需调参即可获得高精度结果，适合需稳健决策的系统
风险敏感规划旨在马尔可夫决策过程（MDPs）中找到最大化尾部指标的策略。这类优化任务对最广泛使用且可解释性高的指标（如阈值概率或（条件）风险价值）而言代价高昂。已有研究表明，唯有熵风险度量（EntRM）可通过动态规划高效优化，但需选择一个难以解释的参数。本文证明，通过对EntRM参数范围内所有最优策略进行计算，可得到目标指标的紧逼近。借助新颖的结构分析与熵风险的光滑性特性，该最优策略前沿可被有效求解。实验证明，该方法在多种决策场景中表现优异。
原文摘要 · Abstract (English)
Risk-sensitive planning aims to identify policies maximizing some tail-focused metrics in Markov Decision Processes (MDPs). Such an optimization task can be very costly for the most widely used and interpretable metrics such as threshold probabilities or (Conditional) Values at Risk. Indeed, previous work showed that only Entropic Risk Measures (EntRM) can be efficiently optimized through dynamic programming, leaving a hard-to-interpret parameter to choose. We show that the computation of the full set of optimal policies for EntRM across parameter values leads to tight approximations for the metrics of interest. We prove that this optimality front can be computed effectively thanks to a novel structural analysis and smoothness properties of entropic risks. Empirical results demonstrate that our approach achieves strong performance in a variety of decision-making scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。