用强化学习让软体三足机器人在仿真中学会走路,成功率超80%。
Development of a PPO-Reinforcement Learned Walking Tripedal Soft-Legged Robot using SOFA
- 基于SOFA仿真框架,用PPO算法训练软体机器人行走。
- 单目标任务成功率达82%,多目标序列导航误差仅19毫米。
- 代码开源,适合对软体机器人与强化学习结合感兴趣的开发者。
刚性机器人研究广泛,而软体机器人仍属探索阶段。在复杂地形和危险环境下,使用软体腿机器人替代人类执行任务具有重要意义。为实现机器人在不同场景中的行为训练,实时物理与视觉仿真至关重要。针对软体机器人,仿真框架仍是难点。本文采用开放仿真架构SOFA,并通过自定义设置充分挖掘其潜力。通过精细调校SOFA参数,成功实现前沿的近端策略优化(PPO)强化学习方法。最终构建了一个可部署的、三足、软腿行走机器人,在SOFA环境中完成稳定行走。导航性能是关键指标:在单目标任务中达到82%的成功率,已属突破;进一步评估多目标序列任务时,累计平方误差偏差仅为19毫米。完整代码已公开于GitHub。
原文摘要 · Abstract (English)
Rigid robots were extensively researched, whereas soft robotics remains an underexplored field. Utilizing soft-legged robots in performing tasks as a replacement for human beings is an important stride to take, especially under harsh and hazardous conditions over rough terrain environments. For the demand to teach any robot how to behave in different scenarios, a real-time physical and visual simulation is essential. When it comes to soft robots specifically, a simulation framework is still an arduous problem that needs to be disclosed. Using the simulation open framework architecture (SOFA) is an advantageous step. However, neither SOFA's manual nor prior public SOFA projects show its maximum capabilities the users can reach. So, we resolved this by establishing customized settings and handling the framework components appropriately. Settling on perfect, fine-tuned SOFA parameters has stimulated our motivation towards implementing the state-of-the-art (SOTA) reinforcement learning (RL) method of proximal policy optimization (PPO). The final representation is a well-defined, ready-to-deploy walking, tripedal, soft-legged robot based on PPO-RL in a SOFA environment. Robot navigation performance is a key metric to be considered for measuring the success resolution. Although in the simulated soft robots case, an 82\% success rate in reaching a single goal is a groundbreaking output, we pushed the boundaries to further steps by evaluating the progress under assigning a sequence of goals. While trailing the platform steps, outperforming discovery has been observed with an accumulative squared error deviation of 19 mm. The full code is publicly available at \href{https://github.com/tarekshohdy/PPO_SOFA_Soft_Legged_Robot.git}{github.com/tarekshohdy/PPO$\textunderscore$SOFA$\textunderscore$Soft$\textunderscore$Legged$\textunderscore$ Robot.git}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。