用大模型自动生成需求访谈后续问题,效果不输人工,还能更好纠正常见错误。
Requirements Elicitation Follow-Up Question Generation
- 基于常见访谈错误类型,让大模型实时生成追问问题。
- 在清晰度、相关性和信息量上,模型生成问题不逊于人类。
- 当引导模型识别错误类型时,其生成问题表现更优,适合新手使用。
访谈是获取软件系统需求的重要方法,但高效访谈需应对领域陌生、认知负荷重和信息过载等挑战。本文研究利用GPT-4o生成需求访谈中的后续问题,基于常见访谈错误类型构建框架,并提出根据受访者发言生成问题的方法。通过两个对照实验评估:第一,仅提供最小引导时,模型生成问题与人工问题在清晰度、相关性和信息量上无显著差异;第二,当生成过程受错误类型引导时,模型表现优于人工。结果表明,大模型可实时辅助访谈者提升需求获取质量与效率。
原文摘要 · Abstract (English)
Interviews are a widely used technique in eliciting requirements to gather stakeholder needs, preferences, and expectations for a software system. Effective interviewing requires skilled interviewers to formulate appropriate interview questions in real time while facing multiple challenges, including lack of familiarity with the domain, excessive cognitive load, and information overload that hinders how humans process stakeholders' speech. Recently, large language models (LLMs) have exhibited state-of-the-art performance in multiple natural language processing tasks, including text summarization and entailment. To support interviewers, we investigate the application of GPT-4o to generate follow-up interview questions during requirements elicitation by building on a framework of common interviewer mistake types. In addition, we describe methods to generate questions based on interviewee speech. We report a controlled experiment to evaluate LLM-generated and human-authored questions with minimal guidance, and a second controlled experiment to evaluate the LLM-generated questions when generation is guided by interviewer mistake types. Our findings demonstrate that, for both experiments, the LLM-generated questions are no worse than the human-authored questions with respect to clarity, relevancy, and informativeness. In addition, LLM-generated questions outperform human-authored questions when guided by common mistakes types. This highlights the potential of using LLMs to help interviewers improve the quality and ease of requirements elicitation interviews in real time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。