让多智能体通信自动学习,动态优化信息共享方式。
Communication Learning in Multi-Agent Systems from Graph Modeling Perspective
- 将通信架构建模为可学习的图,通过双层优化动态调整连接关系。
- 在多种协作任务中,通信效率提升,且适应不同数量的智能体。
- 引入时间门控机制,按需接收信息,降低资源消耗,适合复杂协作场景。
在众多人工智能应用中,多个智能体协同合作是达成目标的关键。为提升协调能力,常采用分布式通信框架,但无差别信息共享会耗费大量资源,而手动预设通信结构则限制了有效协作。此外,推理阶段通信框架通常保持静态,导致持续高资源消耗——实际上多数情况下仅关键决策需要信息交换。本文提出将智能体间的通信架构视为可学习的图,将其建模为通信图的确定问题,并允许架构参数正常更新,从而需要双层优化。通过图表示的连续松弛与注意力单元,提出的CommFormer方法能以端到端方式通过梯度下降高效优化通信图并同步调整架构参数。此外,为每个智能体引入时间门控机制,使其基于当前观测动态决定是否接收共享信息,从而提升决策效率。在多种协作任务上的大量实验验证了模型在多样化协作场景下的鲁棒性,无论智能体数量如何变化,智能体均能发展出更协调、更复杂的策略。
原文摘要 · Abstract (English)
In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We formulate this problem as the task of determining the communication graph while enabling the architecture parameters to update normally, thus necessitating a bi-level optimization process. Utilizing continuous relaxation of the graph representation and incorporating attention units, our proposed approach, CommFormer, efficiently optimizes the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decision-making efficiency. Extensive experiments on a variety of cooperative tasks substantiate the robustness of our model across diverse cooperative scenarios, where agents are able to develop more coordinated and sophisticated strategies regardless of changes in the number of agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。