提出可有限时间停止的算法，实现最优策略的精确识别。
Towards Blackwell Optimality: Bellman Optimality Is All You Can Get
- 基于逐层最优性构造渐近无错学习算法
- 唯一贝尔曼最优策略的MDP类可在有限步内停止
- 适用于需精确最优策略的强化学习研究者
尽管平均收益最优是马尔可夫决策过程（MDPs）中常用的性能度量，但其过于依赖长期行为。引入即时损失度量后，可形成从偏差最优到布莱克威尔最优的层次结构。本文研究此类最优策略的识别问题：针对每阶最优性，构建了错误概率趋于零的学习算法；并刻画了可有限时间内停止识别的MDP类别——即具有唯一贝尔曼最优策略的MDP，该性质与最优性层级无关；最后提供一个可计算的停止规则，与算法结合后，在可终止时必能在有限步内触发。
原文摘要 · Abstract (English)
Although average gain optimality is a commonly adopted performance measure in Markov Decision Processes (MDPs), it is often too asymptotic. Further incorporating measures of immediate losses leads to the hierarchy of bias optimalities, all the way up to Blackwell optimality. In this paper, we investigate the problem of identifying policies of such optimality orders. To that end, for each order, we construct a learning algorithm with vanishing probability of error. Furthermore, we characterize the class of MDPs for which identification algorithms can stop in finite time. That class corresponds to the MDPs with a unique Bellman optimal policy, and does not depend on the optimality order considered. Lastly, we provide a tractable stopping rule that when coupled to our learning algorithm triggers in finite time whenever it is possible to do so.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。