提升多智能体强化学习在故障下的鲁棒性，通过注意力机制和优先采样实现。
Towards Fault Tolerance in Multi-Agent Reinforcement Learning
- 用注意力机制让模型自动识别故障并动态调整关注重点。
- 引入优先采样策略缓解故障前后数据不平衡问题。
- 适合研究多智能体系统容错与可靠性的人参考。
智能体故障严重威胁多智能体强化学习（MARL）算法的性能，带来两大挑战：一是智能体难以从故障引发的状态空间混乱中提取关键信息；二是故障前后的过渡数据在回放缓冲区中分布不均，导致训练样本失衡。本文通过优化模型结构与定制化训练数据采样策略，提升MARL的容错能力。具体而言，在行为者与评论者网络中引入注意力机制，实现故障自动检测并动态调节对故障智能体的关注权重；同时设计优先级机制，选择对当前训练至关重要的过渡数据进行采样。为推动该领域研究，我们开源了一个高度解耦的代码平台，用于故障容忍型MARL实验。实验表明，所提方法能有效应对各类故障、任意智能体发生故障以及随机时间点出现故障的情况。
原文摘要 · Abstract (English)
Agent faults pose a significant threat to the performance of multi-agent reinforcement learning (MARL) algorithms, introducing two key challenges. First, agents often struggle to extract critical information from the chaotic state space created by unexpected faults. Second, transitions recorded before and after faults in the replay buffer affect training unevenly, leading to a sample imbalance problem. To overcome these challenges, this paper enhances the fault tolerance of MARL by combining optimized model architecture with a tailored training data sampling strategy. Specifically, an attention mechanism is incorporated into the actor and critic networks to automatically detect faults and dynamically regulate the attention given to faulty agents. Additionally, a prioritization mechanism is introduced to selectively sample transitions critical to current training needs. To further support research in this area, we design and open-source a highly decoupled code platform for fault-tolerant MARL, aimed at improving the efficiency of studying related problems. Experimental results demonstrate the effectiveness of our method in handling various types of faults, faults occurring in any agent, and faults arising at random times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。