arXiv:2410.16946cs.SEcs.AI2024-10ICLR被引 85

让编程协作网络自动进化,提升软件开发能力

Self-Evolving Multi-Agent Collaboration Networks for Software Development

  • 通过文本反向传播实现多智能体网络自演化
  • 在复杂需求任务上超越现有最佳方法
  • 适合研究自动化软件开发与智能体协作的读者

基于大模型的多智能体协作系统在函数级自动编程中表现优异,但严重依赖人工设计,难以适应真实开发的多样化需求。为此,我们提出EvoMAC,一种受传统神经网络训练启发的自演化多智能体协作范式。EvoMAC通过验证输出与目标代理的一致性获取文本环境反馈,并利用新颖的文本反向传播机制更新网络。为将编码能力从函数级拓展至更复杂的软件级开发,我们进一步构建rSDE-Bench——一个面向需求的软件开发基准,包含多样且复杂的软件需求,并支持自动评估需求正确性。实验表明:i) rSDE-Bench的自动需求感知评估与人类评价高度一致,验证其作为软件级编程基准的可靠性;ii) EvoMAC在软件级rSDE-Bench和函数级HumanEval基准上均优于先前最先进方法,展现更强的编码能力。基准数据集可下载于https://yuzhu-cai.github.io/rSDE-Bench/

原文摘要 · Abstract (English)

LLM-driven multi-agent collaboration (MAC) systems have demonstrated impressive capabilities in automatic software development at the function level. However, their heavy reliance on human design limits their adaptability to the diverse demands of real-world software development. To address this limitation, we introduce EvoMAC, a novel self-evolving paradigm for MAC networks. Inspired by traditional neural network training, EvoMAC obtains text-based environmental feedback by verifying the MAC network's output against a target proxy and leverages a novel textual backpropagation to update the network. To extend coding capabilities beyond function-level tasks to more challenging software-level development, we further propose rSDE-Bench, a requirement-oriented software development benchmark, which features complex and diverse software requirements along with automatic evaluation of requirement correctness. Our experiments show that: i) The automatic requirement-aware evaluation in rSDE-Bench closely aligns with human evaluations, validating its reliability as a software-level coding benchmark. ii) EvoMAC outperforms previous SOTA methods on both the software-level rSDE-Bench and the function-level HumanEval benchmarks, reflecting its superior coding capabilities. The benchmark can be downloaded at https://yuzhu-cai.github.io/rSDE-Bench/.

多智能体自演化软件开发LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。