用强化学习让多智能体在流体中高效会合，突破传统导航局限。
Multi-agent rendezvous in fluid flows via reinforcement learning
- 用多智能体强化学习学习利用流体涡旋特性实现会合
- 相比盲目靠近，会合成功率显著提升，且跨不同涡旋条件可迁移
- 发现弱变形区更适合规划会合点，适合复杂流场中的群体协同研究
多智能体会合是群体系统的关键任务，要求智能体在未指定位置协同汇聚。然而，在流体环境中实现该任务极具挑战，因尚不明确如何利用流体运动学促进收敛。本文采用多智能体强化学习（MARL）方法，在涡旋流中设计物理感知的会合策略。相较于直接朝对方移动的朴素策略，MARL策略显著提升会合成功率，并展现出对涡旋强度、尺度及群组规模变化的良好迁移能力。通过打破状态-动作映射的对称性，该策略利用非直观机制，避免智能体被困于不同涡旋中，从而提高会合成功率。此外，从学习策略中提取的启发式规则也优于朴素策略。理论分析表明，流体拉伸会阻碍会合过程；大有限时间李雅普诺夫指数可识别出使相邻智能体分离的区域，提示应选择弱变形区域规划会合目标。研究揭示了智能体-流体交互在多智能体任务中的关键作用，凸显了MARL在复杂流场中探索群体智能的潜力。
原文摘要 · Abstract (English)
Rendezvous is a critical task for multi-agent systems, requiring agents to coordinate to meet at an unspecified location. However, achieving this in fluid environments presents a challenge, as it remains unclear how agents can exploit underlying fluid kinematics to facilitate convergence. In this study, we adopt a multi-agent reinforcement learning (MARL) approach to develop physics-informed rendezvous strategies in vortical flows. Compared to a naive strategy, where agents navigate toward their counterparts, MARL strategies significantly improve the rendezvous rate. MARL strategies also show transferability across varying vortex intensities, vortex scales, and swarm sizes. By breaking the symmetry of the state-action map, MARL strategy leverages a non-intuitive mechanism that prevents agents from becoming trapped in separate vortices, thereby enhancing rendezvous success. Additionally, a heuristic strategy is extracted from the learned strategy and also outperforms the naive strategy. Furthermore, a theoretical analysis demonstrates that fluid deformation impedes the rendezvous process. Large finite-time Lyapunov exponents identify where fluid effects separate adjacent agents, suggesting that targets should be planned in weak-deformation regions. Our findings reveal the important role that agent-fluid interactions play in multi-agent tasks and highlight the MARL capability to explore swarm intelligence in complex flow environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。