arXiv:2605.30313cs.RO2026-05被引 2

打破机器人强化学习依赖GPU仿真的惯例,实现更高效跨平台训练。

UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms

论文配图:UniLab: A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms
图 1 · 摘自论文原文
  • 分离CPU仿真与GPU学习,通过统一调度提升全流程效率
  • 相同硬件下训练速度提升3到10倍,降低对CUDA依赖
  • 支持苹果Mac、AMD ROCm、Intel XPU等多平台运行

当前基于仿真的机器人强化学习普遍采用以GPU为中心的架构:物理仿真、轨迹采集和学习均部署在单一GPU执行路径上。该范式虽大幅提升训练速度,却导致人们默认高效训练必须依赖GPU进行物理仿真。本文重新审视这一假设,提出:在仿真主导的机器人控制中,关键问题并非由哪个处理器运行物理仿真,而是仿真吞吐量、策略学习与运行时同步能否构成高效的端到端闭环。为此,我们设计了UniLab——一种异构的CPU仿真 / GPU学习架构,通过统一的运行时机制管理数据传输、缓冲与同步,解耦CPU并行仿真与GPU策略更新。UniLab基于MuJoCoUni与MotrixSim CPU批处理物理后端,支持PPO、FastSAC、FlashSAC与APPO算法。在典型仿真机器人控制任务中,同等硬件条件下,其端到端训练效率提升3至10倍,同时减少对NVIDIA CUDA软件栈的依赖,支持在Apple macOS、AMD ROCm及Intel XPU加速器后端上的跨平台执行。结果表明,虽然GPU仿真是高效训练的有效路径,但并非唯一选择,显著拓宽了机器人强化学习训练系统的实际可行性。

原文摘要 · Abstract (English)

Simulation-based RL for contemporary robot control is increasingly organized around GPU-resident simulation: physics, rollout collection, and learning are placed on a single GPU-centric execution path. This paradigm has greatly improved training speed, but it has also encouraged a default assumption that efficient training requires physics to reside on the GPU. We revisit this assumption. Our view is that, in simulation-dominated robot control, the essential question is not which processor runs physics, but whether simulation throughput, policy learning, and runtime synchronization form an efficient end-to-end loop. We present UniLab, a heterogeneous CPU-simulation / GPU-learning architecture that decouples CPU-parallel simulation from GPU policy updates through a unified runtime for data movement, buffering, and synchronization. UniLab is implemented as a complete and extensible training system using MuJoCoUni and MotrixSim CPU-batched physics backends, supporting PPO, FastSAC, FlashSAC, and APPO. On representative simulation-based robot control tasks, UniLab improves end-to-end training efficiency by 3--10$\times$ under the same hardware configuration, while reducing dependence on the NVIDIA CUDA-based software stack and supporting cross-platform execution on the Apple macOS platform and the AMD ROCm and Intel XPU accelerator backends. These results show that GPU simulation is an effective path to efficient training, but not a necessary one, broadening the practical system choices available for robot RL training. Project page: https://unilabsim.github.io.

机器人强化学习异构计算跨平台训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。