分阶段设计奖励函数，让机器人更轻松完成高难度体操动作。
Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approach
- 将复杂任务拆成多个阶段，每阶段定义独立奖励与约束。
- 在仿真和真实环境中均实现比现有方法更好的体操动作表现。
- 适合需要多阶段、多目标控制的机器人强化学习任务。
随着强化学习（RL）处理任务复杂度的提升，奖励函数的设计也日益复杂。本文提出一种简化奖励构建过程的方法：在约束型多目标强化学习（CMORL）框架下，不再使用单一的复合奖励函数，而是为每个任务阶段分别定义多个奖励与代价函数。针对包含顺序性复杂动作的任务，将整个动作序列划分为若干阶段，各阶段独立设置奖励与约束。最后，提出一种实用的CMORL算法，在最大化目标奖励的同时满足由代价函数定义的约束。该方法已在多种仿真与真实机器人环境中的体操类任务中成功验证，表现优于现有的强化学习及约束强化学习算法。代码已开源：https://github.com/rllab-snu/Stage-Wise-CMORL。
原文摘要 · Abstract (English)
As the complexity of tasks addressed through reinforcement learning (RL) increases, the definition of reward functions also has become highly complicated. We introduce an RL method aimed at simplifying the reward-shaping process through intuitive strategies. Initially, instead of a single reward function composed of various terms, we define multiple reward and cost functions within a constrained multi-objective RL (CMORL) framework. For tasks involving sequential complex movements, we segment the task into distinct stages and define multiple rewards and costs for each stage. Finally, we introduce a practical CMORL algorithm that maximizes objectives based on these rewards while satisfying constraints defined by the costs. The proposed method has been successfully demonstrated across a variety of acrobatic tasks in both simulation and real-world environments. Additionally, it has been shown to successfully perform tasks compared to existing RL and constrained RL algorithms. Our code is available at https://github.com/rllab-snu/Stage-Wise-CMORL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。