多无人机在未知环境中用PPO算法实现无碰撞协同探索
On-policy Actor-Critic Reinforcement Learning for Multi-UAV Exploration
- 采用PPO强化学习与CNN+LSTM网络进行分布式自主探索
- 相比PG和A3C，探索覆盖率提升且避免碰撞成功率更高
- 适用于动态未知环境，对新地图具有良好泛化能力
无人机在精准农业、搜救和遥感等领域应用广泛，但在未知环境中自主探索仍具挑战。本文提出基于近端策略优化（PPO）的在线策略强化学习方法，实现多无人机在二维区域内的分布式探索。无人机需避免障碍物及彼此间碰撞，同时通过深度卷积神经网络（CNN）与长短期记忆网络（LSTM）构建的演员-评论家网络，识别已覆盖区域。仿真结果表明，相较策略梯度（PG）与异步优势演员-评论家（A3C）等方法，所提PPO方案在探索效率与安全性上更具优势。结合CNN与LSTM的评论家结构进一步提升了探索性能。此外，该系统在未训练过的地图上仍能完成有效覆盖，展现出良好的泛化能力。实验还分析了超参数调整对整体性能的影响。
原文摘要 · Abstract (English)
Unmanned aerial vehicles (UAVs) have become increasingly popular in various fields, including precision agriculture, search and rescue, and remote sensing. However, exploring unknown environments remains a significant challenge. This study aims to address this challenge by utilizing on-policy Reinforcement Learning (RL) with Proximal Policy Optimization (PPO) to explore the {two dimensional} area of interest with multiple UAVs. The UAVs will avoid collision with obstacles and each other and do the exploration in a distributed manner. The proposed solution includes actor-critic networks using deep convolutional neural networks {(CNN)} and long short-term memory (LSTM) for identifying the UAVs and areas that have already been covered. Compared to other RL techniques, such as policy gradient (PG) and asynchronous advantage actor-critic (A3C), the simulation results demonstrate the superiority of the proposed PPO approach. Also, the results show that combining LSTM with CNN in critic can improve exploration. Since the proposed exploration has to work in unknown environments, the results showed that the proposed setup can complete the coverage when we have new maps that differ from the trained maps. Finally, we showed how tuning hyper parameters may affect the overall performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。