用双动作策略提升强化学习从仿真到现实的迁移效果
Dual Action Policy for Robust Sim-to-Real Reinforcement Learning
- 单策略输出两组动作：任务奖励最大化与域适应
- 在仿真中表现优于基线，且不确定性探索增强鲁棒性
- 适合需要高可靠性迁移到真实世界的强化学习应用
本文提出双动作策略（DAP），以解决强化学习中仿真到现实的动态差异问题。DAP 使用单一策略生成两组动作：一组用于模拟环境中最大化任务奖励，另一组通过奖励调整实现领域自适应。这种解耦设计使训练时更易在源域中最大化总体奖励。此外，DAP 在训练中引入基于不确定性的探索机制，提升智能体鲁棒性。实验表明，DAP 在复杂仿真任务中优于基线方法，结合不确定性估计后性能进一步提升。
原文摘要 · Abstract (English)
This paper presents Dual Action Policy (DAP), a novel approach to address the dynamics mismatch inherent in the sim-to-real gap of reinforcement learning. DAP uses a single policy to predict two sets of actions: one for maximizing task rewards in simulation and another specifically for domain adaptation via reward adjustments. This decoupling makes it easier to maximize the overall reward in the source domain during training. Additionally, DAP incorporates uncertainty-based exploration during training to enhance agent robustness. Experimental results demonstrate DAP's effectiveness in bridging the sim-to-real gap, outperforming baselines on challenging tasks in simulation, and further improvement is achieved by incorporating uncertainty estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。