15分钟用一张显卡训练出能跑跳的类人机器人，抗干扰能力强。
Learning Sim-to-Real Humanoid Locomotion in 15 Minutes
- 用简化版离策略算法+少量奖励设计，在数千个并行仿真中稳定训练
- 单张RTX 4090显卡15分钟完成类人机器人行走控制训练
- 适用于高动态随机环境，适合快速验证类人机器人控制方案
大规模并行仿真已将机器人强化学习训练时间从数天缩短至数分钟。然而，由于高维状态空间和领域随机化等因素，实现快速可靠的类人机器人模拟到现实的强化学习仍具挑战。本文提出一种基于离策略算法（FastSAC 和 FastTD3）的简单实用方案，仅需单张RTX 4090 GPU，15分钟即可完成类人机器人行走策略的快速训练。通过精心设计的超参数与极简奖励函数，该方法在数千个并行环境规模下稳定了离策略算法。我们在Unitree G1和Booster T1机器人上展示了在强领域随机化（如动态参数随机、崎岖地形、推力扰动）下的端到端快速学习，并实现了全身人体运动追踪策略的高效训练。相关视频与开源代码见：https://younggyo.me/fastsac-humanoid。
原文摘要 · Abstract (English)
Massively parallel simulation has reduced reinforcement learning (RL) training time for robots from days to minutes. However, achieving fast and reliable sim-to-real RL for humanoid control remains difficult due to the challenges introduced by factors such as high dimensionality and domain randomization. In this work, we introduce a simple and practical recipe based on off-policy RL algorithms, i.e., FastSAC and FastTD3, that enables rapid training of humanoid locomotion policies in just 15 minutes with a single RTX 4090 GPU. Our simple recipe stabilizes off-policy RL algorithms at massive scale with thousands of parallel environments through carefully tuned design choices and minimalist reward functions. We demonstrate rapid end-to-end learning of humanoid locomotion controllers on Unitree G1 and Booster T1 robots under strong domain randomization, e.g., randomized dynamics, rough terrain, and push perturbations, as well as fast training of whole-body human-motion tracking policies. We provide videos and open-source implementation at: https://younggyo.me/fastsac-humanoid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。