用微调大模型辅助生成符合标准的软件需求,提升效率与质量。
ReqBrain: Task-Specific Instruction Tuning of LLMs for AI-Assisted Requirements Generation
- 基于聊天交互,用微调LLM自动生成需求并分类
- 最佳模型Zephyr-7b-beta在BERT评分达89.30%,FRUGAL得分91.20
- 适合软件工程人员快速生成规范需求,支持后续测试与开发
需求获取与规格说明仍是耗时且易出错的人工过程,给现代软件工程带来挑战。现有研究显示大语言模型(LLMs)可用于自动化生成需求以辅助该过程,但如何有效实现仍不明确。本文提出ReqBrain,一个基于微调LLM的AI辅助工具,可自动生成真实且完整的软件需求。软件工程师可通过对话式会话与ReqBrain交互,实现需求自动创建与类型分类。我们构建了一个符合ISO 29148标准的高质量需求数据集,并对五款7B参数的LLM进行微调,以确定最优基础模型。表现最佳的模型Zephyr-7b-beta在生成需求时获得89.30%的BERT分数和91.20的FRUGAL分数。人工评估进一步验证了其有效性。结果表明,经过微调的生成式AI有望显著改善需求获取与规格化流程,为缺陷识别、测试用例生成及敏捷用户故事创建等方向提供拓展可能。
原文摘要 · Abstract (English)
Requirements elicitation and specification remains a labor-intensive, manual process prone to inconsistencies and gaps, presenting a significant challenge in modern software engineering. Emerging studies underscore the potential of employing large language models (LLMs) for automated requirements generation to support requirements elicitation and specification; however, it remains unclear how to implement this effectively. In this work, we introduce ReqBrain, an Al-assisted tool that employs a fine-tuned LLM to generate authentic and adequate software requirements. Software engineers can engage with ReqBrain through chat-based sessions to automatically generate software requirements and categorize them by type. We curated a high-quality dataset of ISO 29148-compliant requirements and fine-tuned five 7B-parameter LLMs to determine the most effective base model for ReqBrain. The top-performing model, Zephyr-7b-beta, achieved 89.30\% Fl using the BERT score and a FRUGAL score of 91.20 in generating authentic and adequate requirements. Human evaluations further confirmed ReqBrain's effectiveness in generating requirements. Our findings suggest that generative Al, when fine-tuned, has the potential to improve requirements elicitation and specification, paving the way for future extensions into areas such as defect identification, test case generation, and agile user story creation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。