低成本物理摆锤装置助力真实世界强化学习实验
Low-cost Real-world Implementation of the Swing-up Pendulum for Deep Reinforcement Learning Experiments
- 用现成零件搭建低成本摆锤系统，支持真实环境下的强化学习研究
- 可精确测量感知、通信、执行等环节的延迟，分析仿真到现实的差距
- 适合教学与科研，降低物理实验门槛，促进真实世界AI应用
深度强化学习（DRL）在虚拟环境中表现优异，但在真实世界中因仿真与现实的差异而受限。本文介绍了一种低成本的物理倒立摆装置及配套软件环境，旨在帮助研究人员缩小‘仿真到现实’的差距。该装置设计可详细分析物理系统中感知、通信、学习、推断和执行等环节产生的延迟。我们采用常见的商用电子元件、机电部件和传感器，结合标准金属型材、圆棒和3D打印连接件，实现经济高效的物理平台。该实体装置配备基于高保真物理引擎和OpenAI Gym接口的仿真环境，便于对比分析，推动真实世界DRL方法的研究与教学。
原文摘要 · Abstract (English)
Deep reinforcement learning (DRL) has had success in virtual and simulated domains, but due to key differences between simulated and real-world environments, DRL-trained policies have had limited success in real-world applications. To assist researchers to bridge the \textit{sim-to-real gap}, in this paper, we describe a low-cost physical inverted pendulum apparatus and software environment for exploring sim-to-real DRL methods. In particular, the design of our apparatus enables detailed examination of the delays that arise in physical systems when sensing, communicating, learning, inferring and actuating. Moreover, we wish to improve access to educational systems, so our apparatus uses readily available materials and parts to reduce cost and logistical barriers. Our design shows how commercial, off-the-shelf electronics and electromechanical and sensor systems, combined with common metal extrusions, dowel and 3D printed couplings provide a pathway for affordable physical DRL apparatus. The physical apparatus is complemented with a simulated environment implemented using a high-fidelity physics engine and OpenAI Gym interface.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。