通过潜空间建模揭示模型强化学习中仿真到现实的迁移难题
Revealing the Challenges of Sim-to-Real Transfer in Model-Based Reinforcement Learning via Latent Space Modeling
- 用潜空间分析仿真环境对策略性能的影响机制
- 在MuJoCo上验证了该方法可有效度量并缓解仿真到现实的差距
- 适合关注机器人控制与真实部署的模型强化学习研究者
强化学习在机器人控制和自动驾驶等领域日益重要，但仿真环境与真实世界之间的差距仍是实际应用的主要障碍。在模拟环境中训练的智能体往往在真实物理环境中表现下降。本文提出一种基于潜空间的方法，用于分析模型强化学习中仿真对真实策略提升的影响。作为模型基础方法的自然延伸，该方法能直观揭示模型方法在仿真到现实迁移中面临的核心挑战。在MuJoCo环境中的实验评估了该方法在度量和缓解仿真-现实差距方面的效果，并揭示了当前克服该差距仍存在的多种挑战，尤其是针对模型基础方法。
原文摘要 · Abstract (English)
Reinforcement learning (RL) is playing an increasingly important role in fields such as robotic control and autonomous driving. However, the gap between simulation and the real environment remains a major obstacle to the practical deployment of RL. Agents trained in simulators often struggle to maintain performance when transferred to real-world physical environments. In this paper, we propose a latent space based approach to analyze the impact of simulation on real-world policy improvement in model-based settings. As a natural extension of model-based methods, our approach enables an intuitive observation of the challenges faced by model-based methods in sim-to-real transfer. Experiments conducted in the MuJoCo environment evaluate the performance of our method in both measuring and mitigating the sim-to-real gap. The experiments also highlight the various challenges that remain in overcoming the sim-to-real gap, especially for model-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。