用导师机制提升强化学习控制精度，解决长期误差问题。
Adviser-Actor-Critic: Eliminating Steady-State Error in Reinforcement Learning Control
- 引入导师模块指导智能体优化控制动作，融合反馈控制与强化学习优势。
- 在目标导向任务中显著降低稳态误差，性能优于标准RL算法。
- 适合机器人等高精度控制场景，对鲁棒性要求高的应用尤为适用。
高精度控制任务对强化学习算法构成严峻挑战，常因网络近似误差和样本质量不足导致次优表现，尤其在需精确到达目标状态的任务中更为明显，如机器人控制等实际应用。本文提出顾问-执行者-评论家（Adviser-Actor-Critic, AAC）框架，通过结合反馈控制理论的精度与强化学习的自适应能力，引入一个导师模块指导执行者精炼控制动作，从而提升目标达成的精度。在基准测试中，AAC在多个精密、目标条件化的任务上超越标准强化学习算法，展现出更高的精度、可靠性和鲁棒性。代码已公开：https://anonymous.4open.science/r/Adviser-Actor-Critic-8AC5。
原文摘要 · Abstract (English)
High-precision control tasks present substantial challenges for reinforcement learning (RL) algorithms, frequently resulting in suboptimal performance attributed to network approximation inaccuracies and inadequate sample quality.These issues are exacerbated when the task requires the agent to achieve a precise goal state, as is common in robotics and other real-world applications.We introduce Adviser-Actor-Critic (AAC), designed to address the precision control dilemma by combining the precision of feedback control theory with the adaptive learning capability of RL and featuring an Adviser that mentors the actor to refine control actions, thereby enhancing the precision of goal attainment.Finally, through benchmark tests, AAC outperformed standard RL algorithms in precision-critical, goal-conditioned tasks, demonstrating AAC's high precision, reliability, and robustness.Code are available at: https://anonymous.4open.science/r/Adviser-Actor-Critic-8AC5.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。