arXiv:2409.07449cs.RO2024-09被引 5

用深度强化学习训练挖运机自动装矿,真实场景下装满率71%-94%。

Autonomous loading of ore piles with Load-Haul-Dump machines using Deep Reinforcement Learning

  • 在仿真中用强化学习训练挖运机装矿策略,动作空间分连续与混合两种
  • 真实测试中装满率71%-94%,轮子打滑比基线少
  • 对矿堆测量误差有鲁棒性,适合工业自动化场景

本文提出一种基于深度强化学习的控制器训练方法,用于实现载荷-运输-卸载(LHD)机械在矿堆上的自主装料。控制器需完成完整装料动作,在避免轮子打滑、物料洒落或陷入矿堆的前提下填满铲斗。训练完全在仿真环境中进行,采用基于土方工程力学基本方程的简化环境以降低计算成本。训练了两种策略:一种具有混合动作空间,另一种为连续动作空间。强化学习策略在仿真和真实世界中均进行了评估,使用缩比LHD和缩比矿堆,性能与基于启发式规则的控制器及人工遥控对比。额外实验评估了策略对矿堆特征测量误差的鲁棒性。总体而言,强化学习控制器在真实场景中表现良好,装满率介于71%-94%,且装料过程中轮子打滑程度低于其他基线。相关训练环境、模拟行为视频及真实实验视频可查看https://youtu.be/jOpA1rkwhDY。

原文摘要 · Abstract (English)

This work presents a deep reinforcement learning-based approach to train controllers for the autonomous loading of ore piles with a Load-Haul-Dump (LHD) machine. These controllers must perform a complete loading maneuver, filling the LHD's bucket with material while avoiding wheel drift, dumping material, or getting stuck in the pile. The training process is conducted entirely in simulation, using a simple environment that leverages the Fundamental Equation of Earth-Moving Mechanics so as to achieve a low computational cost. Two different types of policies are trained: one with a hybrid action space and another with a continuous action space. The RL-based policies are evaluated both in simulation and in the real world using a scaled LHD and a scaled muck pile, and their performance is compared to that of a heuristics-based controller and human teleoperation. Additional real-world experiments are performed to assess the robustness of the RL-based policies to measurement errors in the characterization of the piles. Overall, the RL-based controllers show good performance in the real world, achieving fill factors between 71-94%, and less wheel drift than the other baselines during the loading maneuvers. A video showing the training environment and the learned behavior in simulation, as well as some of the performed experiments in the real world, can be found in https://youtu.be/jOpA1rkwhDY.

强化学习自动驾驶矿山机器人仿真实验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。