arXiv:2411.13365cs.AIcs.LG2024-11中稿 · AAMAS 2026

用决策树+梅尔机表示有限记忆策略,让难懂的决策过程变清晰。

Explainable Representation of Finite-Memory Policies for POMDPs using Decision Trees

  • 用决策树描述策略的静态部分,梅尔机控制状态切换
  • 相比传统方法,表示规模更小,解释性显著提升
  • 适用于多种有限记忆策略,特别适合吸引子类策略

部分可观测马尔可夫决策过程(POMDPs)是不确定环境下决策的核心框架。由于最优策略通常需要无限记忆,难以实现且多数问题不可判定,因此常采用有限记忆策略。然而,现有计算方法复杂,生成的策略也难以理解。为此,本文提出一种可解释的策略表示方法:结合梅尔机与决策树,前者用于状态切换,后者描述策略的静态行为。我们设计了从标准有限状态控制器(FSC)形式到该表示的转换,并证明其可推广至其他有限记忆策略变体。进一步发现,近期流行的“吸引子基”策略具有特殊结构,可构造更简单紧凑的表示。通过案例研究验证了该方法在可解释性上的优势。

原文摘要 · Abstract (English)

Partially Observable Markov Decision Processes (POMDPs) are a fundamental framework for decision-making under uncertainty and partial observability. Since in general optimal policies may require infinite memory, they are hard to implement and often render most problems undecidable. Consequently, finite-memory policies are mostly considered instead. However, the algorithms for computing them are typically very complex, and so are the resulting policies. Facing the need for their explainability, we provide a representation of such policies, both (i) in an interpretable formalism and (ii) typically of smaller size, together yielding higher explainability. To that end, we combine models of Mealy machines and decision trees; the latter describing simple, stationary parts of the policies and the former describing how to switch among them. We design a translation for policies of the finite-state-controller (FSC) form from standard literature and show how our method smoothly generalizes to other variants of finite-memory policies. Further, we identify specific properties of recently used "attractor-based" policies, which allow us to construct yet simpler and smaller representations. Finally, we illustrate the higher explainability in a few case studies.

POMDP策略解释决策树有限记忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。