用进化算法生成简洁可解释的智能体控制规则,适合安全关键场景。
BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies
- 通过在线进化搜索生成符号化规则序列,保证策略透明可读。
- 在三个基准任务中,规则数量少于深度强化学习基线,性能相当。
- 支持规则数约束与表达力平衡,适合需要人机协同的系统。
可解释强化学习是安全关键应用中自主决策系统部署的重大挑战。现代深度强化学习虽强大,但常产生黑箱策略,影响验证、降低透明度并阻碍人工监管。为此,本文提出BASIL(最佳动作符号可解释学习),一种基于质量-多样性优化的在线进化搜索方法,生成符号化规则策略。策略以状态变量的符号谓词有序列表表示,确保完全可解释性与可控复杂度。利用质量-多样性存档,促进高性能解之间的行为与结构多样性;复杂度感知适应度函数推动紧凑表示的合成。进化系统支持规则数量精确约束与系统可适应性,实现透明性与表达力的平衡。在CartPole-v1、MountainCar-v0和Acrobot-v1三个基准任务上的实证比较表明,BASIL始终能生成与深度强化学习基线性能相当、且更紧凑的可解释控制器。该研究提出一种统一框架,融合符号表达、进化多样性与在线学习,实现可解释策略的高效合成。
原文摘要 · Abstract (English)
The quest for interpretable reinforcement learning is a grand challenge for the deployment of autonomous decision-making systems in safety-critical applications. Modern deep reinforcement learning approaches, while powerful, tend to produce opaque policies that compromise verification, reduce transparency, and impede human oversight. To address this, we introduce BASIL (Best-Action Symbolic Interpretable Learning), a systematic approach for generating symbolic, rule-based policies via online evolutionary search with quality-diversity (QD) optimization. BASIL represents policies as ordered lists of symbolic predicates over state variables, ensuring full interpretability and tractable policy complexity. By using a QD archive, the methodology in the proposed study encourages behavioral and structural diversity between top-performing solutions, while a complexity-aware fitness encourages the synthesis of compact representations. The evolutionary system supports the use of exact constraints for rule count and system adaptability for balancing transparency with expressiveness. Empirical comparisons with three benchmark tasks CartPole-v1, MountainCar-v0, and Acrobot-v1 show that BASIL consistently synthesizes interpretable controllers with compact representations comparable to deep reinforcement learning baselines. Herein, this article introduces a new interpretable policy synthesis method that combines symbolic expressiveness, evolutionary diversity, and online learning through a unifying framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。