用大模型自动识别软件需求间的依赖关系,准确率超93%。
Automating the Detection of Requirement Dependencies Using Large Language Models
- 基于检索增强生成与上下文学习,直接从自然语言需求中识别多种依赖类型。
- 准确率达93%,非依赖项分类F1达0.96,细粒度依赖检测提升超94%。
- 适用于需求工程、自动化测试和系统演化,尤其适合复杂系统团队。
需求之间存在多种类型的依赖关系,其识别对软件开发决策至关重要。然而,由于需求数量庞大、语义模糊且频繁变更,传统方法难以应对。大语言模型(LLM)在自然语言处理方面表现突出,但其在需求依赖检测中的应用尚未深入探索。本文提出LEREDD,一种基于大模型的自动化依赖检测方法,融合检索增强生成(RAG)与上下文学习(ICL),可直接从自然语言需求中识别多样依赖类型。实验表明,该方法准确率达到0.93,F1得分为0.84,非依赖项的平均F1高达0.96。在细粒度依赖检测上,相较于基线模型,其在“Requires”类依赖上的F1得分相对提升分别为94.87%和105.41%。研究还公开了一个包含813个需求对的标注数据集,覆盖三个不同系统,以支持复现与后续研究。
原文摘要 · Abstract (English)
Requirements are inherently interconnected through various types of dependencies. Identifying these dependencies is essential, as they underpin critical decisions and influence a range of activities throughout software development. However, this task is challenging, particularly in modern software systems, given the high volume of complex, coupled requirements. These challenges are further exacerbated by the ambiguity of Natural Language (NL) requirements and their constant change. Consequently, requirement dependency detection is often overlooked or performed manually. Large Language Models (LLMs) exhibit strong capabilities in NL processing, presenting a promising avenue for requirement-related tasks. While they have shown to enhance various requirements engineering tasks, their effectiveness in identifying requirement dependencies remains unexplored. In this paper, we introduce LEREDD, an LLM-based approach for automated detection of requirement dependencies that leverages Retrieval-Augmented Generation (RAG) and In-Context Learning (ICL). It is designed to identify diverse dependency types directly from NL requirements. We empirically evaluate LEREDD against two state-of-the-art baselines. The results show that LEREDD provides highly accurate classification of dependent and non-dependent requirements, achieving an accuracy of 0.93, and an F1 score of 0.84, with the latter averaging 0.96 for non-dependent cases. LEREDD outperforms zero-shot LLMs and baselines, particularly in detecting fine-grained dependency types, where it yields average relative gains of 94.87% and 105.41% in F1 scores for the Requires dependency over the baselines. We also provide an annotated dataset of requirement dependencies encompassing 813 requirement pairs across three distinct systems to support reproducibility and future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。