让工程师实时干预AI写代码，提升开发效率
Human-In-the-Loop Software Development Agents
- 工程师可实时修正AI生成的代码计划和源码
- 实测可减少简单任务的开发时间和精力
- 适合希望提升编码效率的开发团队
近年来，基于大语言模型（LLMs）的多智能体范式被引入软件工程，用于自动完成开发任务（如从问题描述生成源码）。然而，现有工作主要依赖历史基准数据集评估，很少在自动化开发流程中引入人类反馈，也未在实际场景中部署。本文提出一种人机协同的基于LLM的开发框架HULA，使软件工程师能在生成编码计划和源码时实时干预和引导。我们设计、实现并将在Atlassian JIRA中部署了HULA框架。通过多阶段评估发现，Atlassian工程师认为HULA能显著减少整体开发时间与精力，尤其在制定编码计划和编写简单任务代码时效果明显。但部分情况下仍存在代码质量隐患。本文总结经验教训，并探讨未来方向，为推动基于LLM的开发智能体在实践中落地提供支持。
原文摘要 · Abstract (English)
Recently, Large Language Models (LLMs)-based multi-agent paradigms for software engineering are introduced to automatically resolve software development tasks (e.g., from a given issue to source code). However, existing work is evaluated based on historical benchmark datasets, rarely considers human feedback at each stage of the automated software development process, and has not been deployed in practice. In this paper, we introduce a Human-in-the-loop LLM-based Agents framework (HULA) for software development that allows software engineers to refine and guide LLMs when generating coding plans and source code for a given task. We design, implement, and deploy the HULA framework into Atlassian JIRA for internal uses. Through a multi-stage evaluation of the HULA framework, Atlassian software engineers perceive that HULA can minimize the overall development time and effort, especially in initiating a coding plan and writing code for straightforward tasks. On the other hand, challenges around code quality remain a concern in some cases. We draw lessons learned and discuss opportunities for future work, which will pave the way for the advancement of LLM-based agents in software development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。