用树模型替代神经网络，让强化学习决策更透明可解释
Upside-Down Reinforcement Learning for More Interpretable Optimal Control
- 将动作预测转为监督学习，用随机森林等树模型替代神经网络
- 在多个控制基准上表现媲美神经网络，性能差距小于3%
- 树模型天生可解释，适合安全关键场景的强化学习应用
无模型强化学习算法通常学习状态到预期奖励的映射或寻找能最大化性能的策略。基于模型的算法则试图学习环境的近似模型，并结合规划算法使用。反向强化学习（UDRL）是一种新范式，旨在从状态和期望指令中预测动作，将其视为监督学习问题，并已成功由神经网络解决。本文研究除神经网络外的函数逼近方法是否也能在UDRL框架中应用。我们在多个流行最优控制基准上进行实验，结果表明，如随机森林和极化随机树等树基方法表现可与神经网络相当，且生成的策略具有更高的内在可解释性，从而为实现更透明、安全和鲁棒的强化学习铺平道路。
原文摘要 · Abstract (English)
Model-Free Reinforcement Learning (RL) algorithms either learn how to map states to expected rewards or search for policies that can maximize a certain performance function. Model-Based algorithms instead, aim to learn an approximation of the underlying model of the RL environment and then use it in combination with planning algorithms. Upside-Down Reinforcement Learning (UDRL) is a novel learning paradigm that aims to learn how to predict actions from states and desired commands. This task is formulated as a Supervised Learning problem and has successfully been tackled by Neural Networks (NNs). In this paper, we investigate whether function approximation algorithms other than NNs can also be used within a UDRL framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs with the significant benefit of resulting in policies that are inherently more interpretable than NNs, therefore paving the way for more transparent, safe, and robust RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。