提出稀疏浅层专家混合模型，让强化学习决策更透明且性能接近顶尖水平。
SMOSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks
- 用可解释的专家和路由网络构建稀疏激活的控制策略
- 在6个MuJoCo任务上超越现有可解释方法，逼近非可解释模型性能
- 将路由网络蒸馏为决策树，显著提升决策过程可读性
连续控制任务常涉及高维、动态且非线性的环境。当前最优性能由复杂的闭箱策略实现，虽高效但缺乏透明性。可解释策略虽性能较低，却能支持错误诊断、伦理责任和信任建立。本文提出SMOSE，一种基于顶层1个专家的稀疏激活可解释控制器。该方法结合一组训练为不同基础技能专家的可解释决策单元，以及一个负责任务分配的可解释路由网络。通过先进强化学习算法训练，并采用负载均衡技术确保专家使用公平。随后将路由网络权重蒸馏为决策树，极大提升可解释性。在六个MuJoCo基准环境上的评估表明，SMOSE优于近期可解释基线，并缩小了与非可解释顶尖算法之间的差距。
原文摘要 · Abstract (English)
Continuous control tasks often involve high-dimensional, dynamic, and non-linear environments. State-of-the-art performance in these tasks is achieved through complex closed-box policies that are effective, but suffer from an inherent opacity. Interpretable policies, while generally underperforming compared to their closed-box counterparts, advantageously facilitate transparent decision-making within automated systems. Hence, their usage is often essential for diagnosing and mitigating errors, supporting ethical and legal accountability, and fostering trust among stakeholders. In this paper, we propose SMOSE, a novel method to train sparsely activated interpretable controllers, based on a top-1 Mixture-of-Experts architecture. SMOSE combines a set of interpretable decisionmakers, trained to be experts in different basic skills, and an interpretable router that assigns tasks among the experts. The training is carried out via state-of-the-art Reinforcement Learning algorithms, exploiting load-balancing techniques to ensure fair expert usage. We then distill decision trees from the weights of the router, significantly improving the ease of interpretation. We evaluate SMOSE on six benchmark environments from MuJoCo: our method outperforms recent interpretable baselines and narrows the gap with noninterpretable state-of-the-art algorithms
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。