分阶段设计奖励函数，让机器人学得更快更稳。
Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
- 先用任务奖励训练探索，再加入行为奖励优化
- 在多个环境中性能超越直接全奖励训练基线
- 重用训练样本提升稳定性，适合复杂任务训练
深度强化学习是机器人控制的有力工具，但实际应用常受限于有效奖励函数的设计。现实任务通常需同时优化多个目标，需精确调节权重以获得期望策略。为此，我们提出两阶段奖励课程方法，将任务目标与行为项解耦。首先，仅使用任务奖励训练智能体以确保有效探索；随后引入包含能耗等辅助行为项的完整奖励。我们分析了多种过渡策略，表明阶段间重用样本对训练稳定性至关重要。在DeepMind Control Suite、ManiSkill3及移动机器人环境（新增行为目标）上验证，该方法简单有效，显著优于直接使用完整奖励的基线，且对奖励权重更具鲁棒性。
原文摘要 · Abstract (English)
Deep Reinforcement Learning is a promising tool for robotic control, yet practical application is often hindered by the difficulty of designing effective reward functions. Real-world tasks typically require optimizing multiple objectives simultaneously, necessitating precise tuning of their weights to learn a policy with the desired characteristics. To address this, we propose a two-stage reward curriculum where we decouple task-specific objectives from behavioral terms. In our method, we first train the agent on a simplified task-only reward function to ensure effective exploration before introducing the full reward that includes auxiliary behavior-related terms such as energy efficiency. Further, we analyze various transition strategies and demonstrate that reusing samples between phases is critical for training stability. We validate our approach on the DeepMind Control Suite, ManiSkill3, and a mobile robot environment, modified to include auxiliary behavioral objectives. Our method proves to be simple yet effective, substantially outperforming baselines trained directly on the full reward while exhibiting higher robustness to specific reward weightings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。