用GPU加速模拟，让水下无人机快速学会复杂动作
MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation
- 基于GPU的高保真仿真，支持多环境并行训练
- 单卡性能比实时模拟快一万倍，可快速训练多种任务
- 兼容主流强化学习框架，适合水下机器人研究者
强化学习（RL）为无人水下航行器（UUVs）通过试错学习最优行为提供了可能。然而，现有模拟器难以高效集成RL方法，限制了训练扩展性和性能。本文提出MarineGym，一种新型仿真框架，利用GPU加速提升UUV强化学习训练效率。该框架在单张GPU上实现比实时模拟快一万倍的性能，支持在多个水下任务上快速训练RL算法。核心特性包括真实动态建模、并行环境执行，以及对PyTorch和TorchRL等主流强化学习框架的兼容性。通过站位保持、圆周跟踪、螺旋跟踪和双纽线跟踪四项任务验证了其有效性。该框架为推进水下机器人强化学习发展，实现复杂动态环境下的高效训练奠定了基础。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) is a promising solution, allowing Unmanned Underwater Vehicles (UUVs) to learn optimal behaviors through trial and error. However, existing simulators lack efficient integration with RL methods, limiting training scalability and performance. This paper introduces MarineGym, a novel simulation framework designed to enhance RL training efficiency for UUVs by utilizing GPU acceleration. MarineGym offers a 10,000-fold performance improvement over real-time simulation on a single GPU, enabling rapid training of RL algorithms across multiple underwater tasks. Key features include realistic dynamic modeling of UUVs, parallel environment execution, and compatibility with popular RL frameworks like PyTorch and TorchRL. The framework is validated through four distinct tasks: station-keeping, circle tracking, helical tracking, and lemniscate tracking. This framework sets the stage for advancing RL in underwater robotics and facilitating efficient training in complex, dynamic environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。