让多智能体自动分解任务，提升协作学习效率。
Learning Symbolic Task Decompositions for Multi-Agent Teams
- 用任务条件架构同时学任务分解和各智能体策略
- 在无先验知识环境下实现高效协作，成功处理依赖动态
- 适合需要自动任务规划的多智能体系统研究者
提升协作式多智能体学习样本效率的一种方法是将整体任务分解为可分配给个体智能体的子任务。本文在奖励机器（reward machines）框架下研究此问题：一种可形式化分解的符号化任务。为应对缺乏环境先验知识的情形，提出一个无需预设分解的框架，仅通过与环境的无模型交互即可学习最优分解。该方法采用任务条件架构，同步学习最优分解及各子任务对应的智能体策略。既避免了人工设计分解的繁琐，又保持了改进信用分配带来的样本效率优势。在多个深度强化学习场景中验证了有效性，结果表明即使在存在智能体间依赖动态的环境中也能成功实现同步多智能体学习，这是以往工作无法达成的。
原文摘要 · Abstract (English)
One approach for improving sample efficiency in cooperative multi-agent learning is to decompose overall tasks into sub-tasks that can be assigned to individual agents. We study this problem in the context of reward machines: symbolic tasks that can be formally decomposed into sub-tasks. In order to handle settings without a priori knowledge of the environment, we introduce a framework that can learn the optimal decomposition from model-free interactions with the environment. Our method uses a task-conditioned architecture to simultaneously learn an optimal decomposition and the corresponding agents' policies for each sub-task. In doing so, we remove the need for a human to manually design the optimal decomposition while maintaining the sample-efficiency benefits of improved credit assignment. We provide experimental results in several deep reinforcement learning settings, demonstrating the efficacy of our approach. Our results indicate that our approach succeeds even in environments with codependent agent dynamics, enabling synchronous multi-agent learning not achievable in previous works.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。