让智能体按角色和子任务动态分工,提升复杂任务协作效率。
MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts

- 通过角色-子任务条件化设计,实现任务结构与参数双层专业化。
- 在三个模型上代码生成任务中整体与步骤级性能均显著提升。
- 适合需要多智能体协同的复杂长序列任务场景。
基于大语言模型的多智能体系统在复杂、长周期任务中展现出强大潜力。然而,现有方法主要依赖粗粒度提示区分,缺乏针对多样化子任务的参数适应,导致智能体间同质化严重,专业化能力不足,制约了复杂任务的表现。为此,我们提出面向任务的多智能体系统 MoRSE,通过(角色, 子任务)条件化机制,在任务结构与参数层面实现双重专业化。首先,将任务分解为依赖感知的有向无环图(DAG),为每个智能体分配特定(角色, 子任务),明确其职责。其次,提出基于原型的语义路由动态混合(角色, 子任务)LoRA专家模块,高效增强共享大模型底座的参数专业化能力。进一步地,为在稀疏奖励下稳定优化专家与路由,设计分层组相对策略优化,两层信用分配机制分离专家更新与路由决策带来的方差,解耦专家质量与路由质量。在三种骨干模型上的代码生成基准测试表明,本方法在整体任务与逐步表现上均有提升,且训练所得专业化能力可泛化至未见的任务类别与领域。
原文摘要 · Abstract (English)
Large language model-based multi-agent systems have recently shown strong potential for complex, long-horizon tasks. However, existing methods mainly rely on coarse prompt-level differentiation without parameter adaptation for diverse subtasks, resulting in insufficient inter-agent heterogeneity and limited specialized capability that bottleneck performance on tasks with complex requirements. To address this, we introduce a Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts (MoRSE) that distinguishes agents with (role, subtask)-conditional specialization at both the task structure and parameter levels. To make agents' responsibility explicit at the task structure level, we formulate a task-oriented multi-agent system that decomposes each task into a dependency-aware Directed Acyclic Graph of subtasks and assigns each agent a specific (role, subtask), introducing task-level specialization across collaborating agents. Additionally, to address the diverse role and subtask parameter adaptation demands, we propose a dynamic Mixture of (role, subtask) LoRA Experts module with a prototype-based semantic router for subtasks, augmenting agents with parameter-level specialization on a shared LLM substrate cost-effectively. Then, to co-optimize experts and router stably under sparse task rewards, we further propose a hierarchical group-relative policy optimization with two-layer credit assignment that isolates expert updates from the cross-route variance introduced by routing decisions, disentangling expert quality from routing quality. Experiments on code-generation benchmarks across three backbones demonstrate the effectiveness of our approach, with improvements in both whole-task and step-wise performance, and the gains from trained specialization generalize across held-out task categories and domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。