arXiv:2409.03260cs.AIcs.LG2024-09AAAI被引 2

无需环境模型,可自动合成最优决策树策略

In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search

  • 基于搜索的决策树构造方法,利用轨迹剪枝大幅缩减搜索空间
  • 在黑箱环境中实现最小步数达标的决策树策略,具备理论最优性保证
  • 适合需要可解释控制策略的系统设计者,如自动驾驶、机器人控制

决策树因其可解释性,是动态系统控制策略的理想选择。然而,构建或合成此类策略极具挑战。以往方法依赖神经网络策略模仿、形式合成生成的表格策略、强化学习或混合整数线性规划,但通常需精确策略或环境形式模型,且无法保证最终树策略的质量或规模。本文提出一种新方法,在给定确定性黑箱环境、规范、树谓词离散化及初始状态集的前提下,合成以最少步数达成目标为优化目标的最优决策树策略。该方法是一种专用搜索算法,系统探索由给定离散化定义的指数级庞大决策树空间。核心创新在于一种基于轨迹的剪枝机制,显著缩小搜索范围。本方法为黑箱环境与黑箱规范下小规模、有最优性保证的决策树策略合成提供了概念上新颖的解决方案。

原文摘要 · Abstract (English)

Decision trees, owing to their interpretability, are attractive as control policies for (dynamical) systems. Unfortunately, constructing, or synthesising, such policies is a challenging task. Previous approaches do so by imitating a neural-network policy, approximating a tabular policy obtained via formal synthesis, employing reinforcement learning, or modelling the problem as a mixed-integer linear program. However, these works may require access to a hard-to-obtain accurate policy or a formal model of the environment (within reach of formal synthesis), and may not provide guarantees on the quality or size of the final tree policy. In contrast, we present an approach to synthesise optimal decision-tree policies given a deterministic black-box environment and specification, a discretisation of the tree predicates, and an initial set of states, where optimality is defined with respect to the number of steps to achieve the goal. Our approach is a specialised search algorithm which systematically explores the (exponentially large) space of decision trees under the given discretisation. The key component is a novel trace-based pruning mechanism that significantly reduces the search space. Our approach represents a conceptually novel way of synthesising small decision-tree policies with optimality guarantees even for black-box environments with black-box specifications.

决策树策略合成黑箱系统搜索算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。