改进MADDPG算法，让多智能体更协作、学习更高效。
Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling
- 引入动作推断机制，预测其他智能体行为以提升策略精度。
- 用几何分布重要性采样，优先回放近期有用经验。
- 在捕食者-猎物任务中显著提升稳定性和探索效率。
我们研究多智能体深度强化学习，提出对多智能体深度确定性策略梯度（MADDPG）算法的两项改进。首先，引入一种新的动作推断机制，使每个智能体能够预测其他智能体的意图动作，从而提升自身策略的准确性和稳定性。其次，采用基于几何分布的重要性采样策略，应用于经验回放缓冲区，优先选择更近期且信息量大的经验，有效缓解多智能体环境中的非平稳性问题。我们在PettingZoo库提供的离散动作捕食者-猎物任务上评估了这两项改进。结果表明，动作推断能显著提升学习稳定性与智能体间协作能力；使用几何分布的重要性采样可大幅提高探索效率，优于标准MADDPG。代码已开源：https://github.com/shaashwathsivakumar/MARL_Proj。
原文摘要 · Abstract (English)
We investigate multi-agent deep reinforcement learning and propose two enhancements to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. First, we introduce a novel Action Inference mechanism that enables each agent to predict other agents' intended actions, thereby improving the accuracy and stability of its own policy. Second, we apply an importance sampling strategy, using geometric distribution, in the replay buffer to prioritize more recent and informative experiences, which helps mitigate the non-stationarity inherent in multi-agent environments. We evaluate both modifications on the discrete-action Predator-Prey task provided by the PettingZoo library, a flexible Python interface for general multi-agent reinforcement learning benchmarks. Our results indicate that Action Inference is effective in improving learning stability and inter-agent cooperation and that importance sampling using geometric distribution can lead to significant improvements in exploration efficiency over standard MADDPG. Code available at https://github.com/shaashwathsivakumar/MARL_Proj
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。