动态调整智能体数量与通信结构,让大模型协作更高效。
Adaptive Graph Pruning for Multi-Agent Communication
- 根据任务自动剪枝智能体数量和通信拓扑,实现软硬协同优化。
- 在6个基准上提升2.58%~9.84%,三类任务表现均优异。
- 训练步数少、耗token少,90%以上减少计算开销,适合部署。
基于大语言模型的多智能体系统在各类任务中表现出色,尤其在协作通信增强后。然而,现有方法通常依赖固定智能体数量和静态通信结构,难以适应任务复杂度变化。本文提出自适应图剪枝(AGP),一种任务自适应的多智能体协作框架,联合优化智能体数量(硬剪枝)与通信拓扑(软剪枝)。具体地,采用两阶段训练策略:首先独立训练不同智能体数量下的软剪枝网络,确定特定任务最优的完整图与位置掩码;随后在最大完整图内联合优化硬剪枝与软剪枝,动态配置每项任务的智能体数量与通信结构。大量实验表明:(1) 性能优越,在六个基准上达到领先水平,跨主流LLM架构一致泛化,性能提升2.58%~9.84%;(2) 任务自适应,动态构建适配特定任务的优化通信拓扑,在一般推理、数学推理与代码生成三类任务中表现极佳;(3) 令牌经济高效,训练步数与令牌消耗显著降低,令牌消耗减少90%以上;(4) 训练高效,仅需约十步训练即可超越现有基线,在六项基准上实现高性能。
原文摘要 · Abstract (English)
Large Language Model (LLM) based multi-agent systems have shown remarkable performance in various tasks, especially when enhanced through collaborative communication. However, current methods often rely on a fixed number of agents and static communication structures, limiting their ability to adapt to varying task complexities. In this paper, we propose Adaptive Graph Pruning (AGP), a novel task-adaptive multi-agent collaboration framework that jointly optimizes agent quantity (hard-pruning) and communication topology (soft-pruning). Specifically, our method employs a two-stage training strategy: firstly, independently training soft-pruning networks for different agent quantities to determine optimal agent-quantity-specific complete graphs and positional masks across specific tasks; and then jointly optimizing hard-pruning and soft-pruning within a maximum complete graph to dynamically configure the number of agents and their communication topologies per task. Extensive experiments demonstrate that our approach is: (1) High-performing, achieving state-of-the-art results across six benchmarks and consistently generalizes across multiple mainstream LLM architectures, with a increase in performance of $2.58\%\sim 9.84\%$; (2) Task-adaptive, dynamically constructing optimized communication topologies tailored to specific tasks, with an extremely high performance in all three task categories (general reasoning, mathematical reasoning, and code generation); (3) Token-economical, having fewer training steps and token consumption at the same time, with a decrease in token consumption of $90\%+$; and (4) Training-efficient, achieving high performance with very few training steps compared with other methods. The performance will surpass the existing baselines after about ten steps of training under six benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。