LLM在需求获取上比人类专家更准更全，速度提升720倍且成本极低。
Analysis of LLMs vs Human Experts in Requirements Engineering
- 在限定时间和提示条件下，让LLM和人类专家分别提取系统需求。
- LLM生成的需求更契合实际（+1.12分），完整性高10.2%，速度达人类720倍。
- 用户误判更像人写的其实是LLM产物，适合快速原型设计与资源紧张项目。
当前关于大语言模型（LLM）在软件开发中的研究多集中于代码生成，而对需求工程（RE）的影响关注较少。本研究在时间与提示受限条件下，对比了LLM与人类专家在需求获取中的表现。结果显示，LLM生成的需求与实际要求的契合度更高（+1.12分），完整性提升10.2%；尽管如此，用户普遍认为更契合的需求应出自人类之手。此外，LLM生成文档的速度达到人类的720倍，平均成本仅为人类专家的0.06%。综合表明，LLM将在需求工程中发挥关键作用，提升需求定义质量，优化资源配置，并缩短项目周期。
原文摘要 · Abstract (English)
The majority of research around Large Language Models (LLM) application to software development has been on the subject of code generation. There is little literature on LLMs' impact on requirements engineering (RE), which deals with the process of developing and verifying the system requirements. Within RE, there is a subdiscipline of requirements elicitation, which is the practice of discovering and documenting requirements for a system from users, customers, and other stakeholders. In this analysis, we compare LLM's ability to elicit requirements of a software system, as compared to that of a human expert in a time-boxed and prompt-boxed study. We found LLM-generated requirements were evaluated as more aligned (+1.12) than human-generated requirements with a trend of being more complete (+10.2%). Conversely, we found users tended to believe that solutions they perceived as more aligned had been generated by human experts. Furthermore, while LLM-generated documents scored higher and performed at 720x the speed, their cost was, on average, only 0.06% that of a human expert. Overall, these findings indicate that LLMs will play an increasingly important role in requirements engineering by improving requirements definitions, enabling more efficient resource allocation, and reducing overall project timelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。