arXiv:2604.22207cs.SEcs.AI2026-04被引 1

用大模型从文档中自动提取软件需求目标,但效果仍需人工辅助。

Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations

  • 设计三阶段流程:识别角色、提取高低层目标,用提示工程驱动多模型协作。
  • 低层目标识别准确率达61%,反馈机制显著提升性能。
  • 适合想加速需求分析的人,不推荐完全替代人工提取。

由于需求工程(RE)文档具有文本密集和重复性强的特点,大语言模型(LLMs)在自动化生成与处理方面已显成效。本文探讨通过三个阶段——角色识别、高层与低层目标提取——实现面向目标的需求工程(GORE)自动化。为此提出基于工程化提示的多模型链式架构,并实验对比了不同上下文学习变体,测量输入数据与示例间的相似性以评估影响。引入双模型反馈循环机制(生成-批判),结果显示零样本反馈优于独立少样本,消融实验证明反馈环对性能至关重要;但少样本结合反馈并未带来提升,暗示主要瓶颈在于‘批判者’模型的提示策略。未来工作将结合检索增强生成(RAG)与思维链(CoT)提示,优化示例数量与质量,进一步提高准确性。

原文摘要 · Abstract (English)

Due to the textual and repetitive nature of many Requirements Engineering (RE) artefacts, Large Language Models (LLMs) have proven useful to automate their generation and processing. In this paper, we discuss a possible approach for automating the Goal-Oriented Requirements Engineering (GORE) process by extracting functional goals from software documentation through three phases: actor identification, high and low-level goal extraction. To implement these functionalities, we propose a chain of LLMs fed with engineered prompts. We experimented with different variants of in-context learning and measured the similarities between input data and in-context examples to better investigate their impact. Another key element is the generation-critic mechanism, implemented as a feedback loop involving two LLMs. Although the pipeline achieved 61% accuracy in low-level goal identification, the final stage, these results indicate the approach is best suited as a tool to accelerate manual extraction rather than as a full replacement. The feedback-loop mechanism with Zero-shot outperformed stand-alone Few-shot, with an ablation study suggesting that performance slightly degrades without the feedback cycle. However, we reported that the combination of the feedback mechanism with Few-shot does not deliver any advantage, possibly suggesting that the primary performance ceiling is the prompting strategy applied to the 'critic' LLM. Together with the refinement of both the quantity and quality of the Shot examples, future research will integrate Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) prompting to improve accuracy.

需求工程大模型应用提示工程自动化提取

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。