用辩论机制优化LLM工作流，提升性能并减少资源消耗
DebFlow: Automating Agent Creation via Agent Debate
- 引入辩论机制让多个代理互评优化工作流
- 平均性能提升3%，训练资源消耗减少37%
- 适合需要高效自动化的工作流设计场景
大型语言模型（LLMs）在自动化生成和优化工作流方面展现出巨大潜力。然而，现有方法存在推理能力有限、计算开销高和资源需求大的问题。为此，我们提出DebFlow框架，通过辩论机制优化工作流，并引入反思机制基于过往经验持续改进。我们在六个基准数据集（包括HotpotQA、MATH和ALFWorld）上评估该方法，相比最新基线平均性能提升3%。训练过程中，资源消耗比当前最优基线降低37%。消融实验表明，移除辩论模块导致两个数据集上性能下降4%，远高于移除反思模块带来的2%下降，凸显辩论在提升性能中的关键作用，同时验证反思的辅助优化价值。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong potential and impressive performance in automating the generation and optimization of workflows. However, existing approaches are marked by limited reasoning capabilities, high computational demands, and significant resource requirements. To address these issues, we propose DebFlow, a framework that employs a debate mechanism to optimize workflows and integrates reflexion to improve based on previous experiences. We evaluated our method across six benchmark datasets, including HotpotQA, MATH, and ALFWorld. Our approach achieved a 3\% average performance improvement over the latest baselines, demonstrating its effectiveness in diverse problem domains. In particular, during training, our framework reduces resource consumption by 37\% compared to the state-of-the-art baselines. Additionally, we performed ablation studies. Removing the Debate component resulted in a 4\% performance drop across two benchmark datasets, significantly greater than the 2\% drop observed when the Reflection component was removed. These findings strongly demonstrate the critical role of Debate in enhancing framework performance, while also highlighting the auxiliary contribution of reflexion to overall optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。