测试离线强化学习在机器人动作扰动下的鲁棒性，发现其比在线方法更脆弱。
Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations
- 用随机与对抗性扰动模拟关节故障，评估离线RL性能
- 离线RL在扰动下平均奖励下降显著，不如在线RL稳定
- 适用于关注机器人控制可靠性的研究者
离线强化学习仅从数据集学习，无需环境交互，对机器人控制极具潜力。然而，其在真实场景中面对关节执行器故障等挑战的鲁棒性仍存疑。本研究基于OpenAI Gym中的四足机器人，通过向关节扭矩信号注入随机和对抗性扰动，模拟最坏情况，评估现有离线强化学习方法的鲁棒性。实验结果显示，现有离线方法对动作扰动极为敏感，性能下降严重，且显著弱于在线强化学习方法，凸显该领域亟需更稳健的算法。
原文摘要 · Abstract (English)
Offline reinforcement learning, which learns solely from datasets without environmental interaction, has gained attention. This approach, similar to traditional online deep reinforcement learning, is particularly promising for robot control applications. Nevertheless, its robustness against real-world challenges, such as joint actuator faults in robots, remains a critical concern. This study evaluates the robustness of existing offline reinforcement learning methods using legged robots from OpenAI Gym based on average episodic rewards. For robustness evaluation, we simulate failures by incorporating both random and adversarial perturbations, representing worst-case scenarios, into the joint torque signals. Our experiments show that existing offline reinforcement learning methods exhibit significant vulnerabilities to these action perturbations and are more vulnerable than online reinforcement learning methods, highlighting the need for more robust approaches in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。