用迭代训练决策树复现强化学习机器人控制，效果媲美原模型且更轻量。
Putting the Iterative Training of Decision Trees to the Test on a Real-World Robotic Task
- 通过迭代采样优化决策树训练，兼顾动作选择能力与状态覆盖
- 在真实机械摆系统上实现与深度强化学习模型相当的平衡性能
- 适合需要透明、低资源强化学习模型的工业落地场景
此前研究中，我们基于深度强化学习（DRL）网络开发了训练决策树（DT）作为强化学习智能体的方法。构建决策树的样本以环境状态为特征，对应动作为标签。为解决样本选取难题——既要反映DRL智能体正确选行动的能力，又要充分覆盖状态空间以保证泛化性，我们提出一种迭代训练算法。本文首次将该算法应用于真实世界的机器人任务。该任务涉及安装于直线轨道小车上的物理摆，目标是通过左右移动使摆杆竖直并保持在不稳定平衡状态。实验结果表明，该算法可生成性能匹配原始DRL智能体的决策树，且参数更少。本研究为从DRL智能体中提取可解释、轻量级的决策树模型提供了起点，适用于真实世界强化学习任务。
原文摘要 · Abstract (English)
In previous research, we developed methods to train decision trees (DT) as agents for reinforcement learning tasks, based on deep reinforcement learning (DRL) networks. The samples from which the DTs are built, use the environment's state as features and the corresponding action as label. To solve the nontrivial task of selecting samples, which on one hand reflect the DRL agent's capabilities of choosing the right action but on the other hand also cover enough state space to generalize well, we developed an algorithm to iteratively train DTs. In this short paper, we apply this algorithm to a real-world implementation of a robotic task for the first time. Real-world tasks pose additional challenges compared to simulations, such as noise and delays. The task consists of a physical pendulum attached to a cart, which moves on a linear track. By movements to the left and to the right, the pendulum is to be swung in the upright position and balanced in the unstable equilibrium. Our results demonstrate the applicability of the algorithm to real-world tasks by generating a DT whose performance matches the performance of the DRL agent, while consisting of fewer parameters. This research could be a starting point for distilling DTs from DRL agents to obtain transparent, lightweight models for real-world reinforcement learning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。