通过残差机制与多样性选择，提升多智能体系统效率与准确性。
RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation
- 引入残差连接与向量相似度筛选，优化智能体响应多样性。
- 在多个基准测试中超越现有方法，计算开销降低40%以上。
- 适合需要高效多任务推理的AI系统开发者使用。
基于大语言模型的多智能体系统虽具备多任务处理能力，但仍受限于高计算开销、信息丢失与鲁棒性不足。受ResNet残差学习启发，我们提出残差混合智能体（RMoA），通过残差连接提升效率与可靠性。为最大化模型响应的信息利用率并最小化计算成本，创新设计基于嵌入的多样性选择机制，采用贪心策略根据向量相似度选取响应。为缓解迭代过程中的信息退化，引入残差提取智能体，捕捉层间响应差异以保留增量信息，并搭配残差聚合智能体实现层次化信息整合。此外，提出自适应终止机制，根据残差收敛情况动态停止处理，进一步提升推理效率。RMoA在对齐、数学推理、代码生成及多任务理解等基准上达到当前最优性能，同时显著降低计算开销。代码已开源：https://github.com/mindhunter01/RMoA。
原文摘要 · Abstract (English)
Although multi-agent systems based on large language models show strong capabilities on multiple tasks, they are still limited by high computational overhead, information loss, and robustness. Inspired by ResNet's residual learning, we propose Residual Mixture-of-Agents (RMoA), integrating residual connections to optimize efficiency and reliability. To maximize information utilization from model responses while minimizing computational costs, we innovatively design an embedding-based diversity selection mechanism that greedily selects responses via vector similarity. Furthermore, to mitigate iterative information degradation, we introduce a Residual Extraction Agent to preserve cross-layer incremental information by capturing inter-layer response differences, coupled with a Residual Aggregation Agent for hierarchical information integration. Additionally, we propose an adaptive termination mechanism that dynamically halts processing based on residual convergence, further improving inference efficiency. RMoA achieves state-of-the-art performance on the benchmarks of across alignment, mathematical reasoning, code generation, and multitasking understanding, while significantly reducing computational overhead. Code is available at https://github.com/mindhunter01/RMoA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。