多船协同导航避障，靠智能通信提升协作效率
Multiple Ships Cooperative Navigation and Collision Avoidance using Multi-agent Reinforcement Learning with Communication
- 用带通信的MADDPG算法让多船自主协商策略
- 通信使多船在信息不全时仍能高效避障，性能远超单船算法
- 适合研究多智能体协同控制与海上无人系统的人看
在现实世界中，无人水面艇（USV）常需协同完成任务。然而，由于非平稳性和部分可观测性，多智能体系统的协同控制面临挑战。近年来，多智能体强化学习（MARL）为解决这些问题提供了新思路。为此，我们提出采用带通信机制的多智能体深度确定性策略梯度（MADDPG）算法，解决多船在部分可观测条件下的协作问题。基于OpenAI Gym环境，我们设计了两项任务：协同导航与协同避障。在此过程中，船只不仅需学习有效控制策略，还需与其他智能体建立通信协议。我们分析了外部噪声对通信的影响、智能体间通信对性能的作用，以及智能体学习到的通信模式。结果表明，所提框架能有效实现多艘船只的协同导航与避障，显著优于传统单智能体算法。智能体建立了稳定的通信协议，通过共享观测弥补信息缺失，实现更优协同。
原文摘要 · Abstract (English)
In the real world, unmanned surface vehicles (USV) often need to coordinate with each other to accomplish specific tasks. However, achieving cooperative control in multi-agent systems is challenging due to issues such as non-stationarity and partial observability. Recent advancements in Multi-Agent Reinforcement Learning (MARL) provide new perspectives to address these challenges. Therefore, we propose using the multi-agent deep deterministic policy gradient (MADDPG) algorithm with communication to address multiple ships' cooperation problems under partial observability. We developed two tasks based on OpenAI's gym environment: cooperative navigation and cooperative collision avoidance. In these tasks, ships must not only learn effective control strategies but also establish communication protocols with other agents. We analyze the impact of external noise on communication, the effect of inter-agent communication on performance, and the communication patterns learned by the agents. The results demonstrate that our proposed framework effectively addresses cooperative navigation and collision avoidance among multiple vessels, significantly outperforming traditional single-agent algorithms. Agents establish a consistent communication protocol, enabling them to compensate for missing information through shared observations and achieve better coordination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。