将因果推理引入多智能体强化学习，提升协作效率与可解释性。
A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inference
- 在先进多智能体场景中引入简单因果增强方法
- 验证了因果推理对策略效能和收敛速度的积极影响
- 揭示了多智能体环境下因果应用的关键挑战与研究方向
因果推理在强化学习中日益受到关注，可提升策略效能、收敛效率、泛化能力、行为安全性和可解释性。然而，其在多智能体强化学习（MARL）中的应用仍处于空白。本文首次探索将因果推理应用于MARL的机遇与挑战。我们在需高度协作的先进MARL场景中，测试了简单因果增强的效果，并评估了多种协作程度下主流MARL算法的表现。结果表明，因果推理在部分任务中显著改善了学习性能，但也暴露出适应性不足等问题。研究为未来将因果强化学习成功迁移到多智能体环境提供了关键方向。
原文摘要 · Abstract (English)
Causal reasoning is increasingly used in Reinforcement Learning (RL) to improve the learning process in several dimensions: efficacy of learned policies, efficiency of convergence, generalisation capabilities, safety and interpretability of behaviour. However, applications of causal reasoning to Multi-Agent RL (MARL) are still mostly unexplored. In this paper, we take the first step in investigating the opportunities and challenges of applying causal reasoning in MARL. We measure the impact of a simple form of causal augmentation in state-of-the-art MARL scenarios increasingly requiring cooperation, and with state-of-the-art MARL algorithms exploiting various degrees of collaboration between agents. Then, we discuss the positive as well as negative results achieved, giving us the chance to outline the areas where further research may help to successfully transfer causal RL to the multi-agent setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。