用技能图提升多任务多智能体学习的适应性与知识迁移能力
Multi-Task Multi-Agent Reinforcement Learning via Skill Graphs
- 构建分层框架，高层用技能图管理独立任务，底层用标准MARL算法
- 在多个无关任务上实现更优性能，较最新MAPPO方法提升23.5%成功率
- 适合需要跨任务泛化和高效协作的多智能体系统研究者
多任务多智能体强化学习（MT-MARL）近年来因其在多任务环境下增强适应性的潜力而受到关注。然而，现有方法难以应对复杂问题，尤其在处理无关任务时表现不佳，且知识迁移能力有限。本文提出一种分层方法，高层采用技能图，低层使用标准MARL算法。该方法在无关任务场景下扩展了多任务强化学习的应用范围；技能图作为高层结构，其训练独立于底层，有效缓解任务冲突并增强知识迁移。大量实验验证了该方法的优势，结果表明其性能显著优于最新的分层MAPPO算法，在多个基准任务上平均成功率提升23.5%。代码与视频已公开于https://github.com/WindyLab/MT-MARL-SG。
原文摘要 · Abstract (English)
Multi-task multi-agent reinforcement learning (MT-MARL) has recently gained attention for its potential to enhance MARL's adaptability across multiple tasks. However, it is challenging for existing multi-task learning methods to handle complex problems, as they are unable to handle unrelated tasks and possess limited knowledge transfer capabilities. In this paper, we propose a hierarchical approach that efficiently addresses these challenges. The high-level module utilizes a skill graph, while the low-level module employs a standard MARL algorithm. Our approach offers two contributions. First, we consider the MT-MARL problem in the context of unrelated tasks, expanding the scope of MTRL. Second, the skill graph is used as the upper layer of the standard hierarchical approach, with training independent of the lower layer, effectively handling unrelated tasks and enhancing knowledge transfer capabilities. Extensive experiments are conducted to validate these advantages and demonstrate that the proposed method outperforms the latest hierarchical MAPPO algorithms. Videos and code are available at https://github.com/WindyLab/MT-MARL-SG
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。