用大模型+图结构提升多智能体协作效率与安全
Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy
- 大模型分解复杂任务为可执行子任务,生成动作依赖图
- 图结构策略实现多智能体高效协同,支持新环境快速适应
- 在AI2-THOR上验证了强性能与可扩展性,适合复杂场景
多智能体系统(MAS)在执行复杂任务方面展现出巨大潜力,但协调与安全性仍是重大挑战。多智能体强化学习(MARL)虽为协作提供了良好框架,却难以应对复杂任务并设计有效奖励函数。大语言模型(LLM)为MAS带来了更强的推理与认知能力,但现有基于LLM的系统在动态环境中响应速度与准确性不足。为此,我们提出基于大模型图协作的多智能体强化学习框架(LGC-MARL),高效融合LLM与MARL。该框架将复杂任务分解为可执行子任务,通过图结构实现多智能体间的高效协作。具体包括两个核心组件:一个由大模型驱动的规划器,负责将复杂指令转化为一系列可执行子任务,利用评判模型评估其合理性,并生成动作依赖图;另一个是基于图的协作元策略,根据动作依赖图促进智能体间通信与协作,并通过元学习适应新任务环境。在AI2-THOR仿真平台上的实验表明,LGC-MARL在完成多种复杂任务时表现出卓越性能与良好的可扩展性。
原文摘要 · Abstract (English)
Multi-agent systems (MAS) have shown great potential in executing complex tasks, but coordination and safety remain significant challenges. Multi-Agent Reinforcement Learning (MARL) offers a promising framework for agent collaboration, but it faces difficulties in handling complex tasks and designing reward functions. The introduction of Large Language Models (LLMs) has brought stronger reasoning and cognitive abilities to MAS, but existing LLM-based systems struggle to respond quickly and accurately in dynamic environments. To address these challenges, we propose LLM-based Graph Collaboration MARL (LGC-MARL), a framework that efficiently combines LLMs and MARL. This framework decomposes complex tasks into executable subtasks and achieves efficient collaboration among multiple agents through graph-based coordination. Specifically, LGC-MARL consists of two main components: an LLM planner and a graph-based collaboration meta policy. The LLM planner transforms complex task instructions into a series of executable subtasks, evaluates the rationality of these subtasks using a critic model, and generates an action dependency graph. The graph-based collaboration meta policy facilitates communication and collaboration among agents based on the action dependency graph, and adapts to new task environments through meta-learning. Experimental results on the AI2-THOR simulation platform demonstrate the superior performance and scalability of LGC-MARL in completing various complex tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。