arXiv:2410.00425cs.ROcs.AI2024-10被引 287

打造最快通用机器人仿真平台,支持千倍加速训练。

ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI

  • GPU并行仿真与渲染,全程无Python/PyTorch开销
  • 实测最高达30,000 FPS,训练时间从小时缩短至分钟
  • 覆盖12类场景,含艺术设计与真实数字孪生环境

仿真已推动机器人学习的可扩展计算范式。但多数现有框架仅支持有限场景与任务,缺乏通用机器人学习和仿真到现实迁移的关键特性。我们推出并开源了ManiSkill3,这是目前最快的基于GPU并行化的机器人仿真器,专为高接触物理的通用操作任务设计。它支持仿真+渲染、异构仿真、点云/体素视觉输入等多方面GPU并行。在典型环境中,带渲染的仿真速度比其他平台快10–1000倍,显存占用减少2–3倍,最高可达30,000 FPS,得益于系统中极低的Python/PyTorch开销、全GPU仿真以及SAPIEN并行渲染系统。原本需数小时训练的任务现可在几分钟内完成。我们还提供了最全面的GPU并行化环境与任务,涵盖12个不同领域,包括绘画、移动操作、人形机器人及艺术家设计或真实世界数字孪生中的灵巧操作。此外,包含来自运动规划、强化学习与遥操作的数百万帧示范数据,并提供覆盖主流强化学习与从示范学习算法的完整基准线。

原文摘要 · Abstract (English)

Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks and lack features critical for scaling generalizable robotics and sim2real. We introduce and open source ManiSkill3, the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation. ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more. Simulation with rendering on ManiSkill3 can run 10-1000x faster with 2-3x less GPU memory usage than other platforms, achieving up to 30,000+ FPS in benchmarked environments due to minimal python/pytorch overhead in the system, simulation on the GPU, and the use of the SAPIEN parallel rendering system. Tasks that used to take hours to train can now take minutes. We further provide the most comprehensive range of GPU parallelized environments/tasks spanning 12 distinct domains including but not limited to mobile manipulation for tasks such as drawing, humanoids, and dextrous manipulation in realistic scenes designed by artists or real-world digital twins. In addition, millions of demonstration frames are provided from motion planning, RL, and teleoperation. ManiSkill3 also provides a comprehensive set of baselines that span popular RL and learning-from-demonstrations algorithms.

机器人仿真GPU加速强化学习数字孪生

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。