用大模型把杂乱应用评论转成可用用户故事，提升需求开发效率。
From Reviews to Requirements: Can LLMs Generate Human-Like User Stories?
- 用少量示例提示让大模型从评论生成用户故事
- 模型生成的故事流畅度超过人类，但重复率高
- 适合敏捷开发团队快速转化用户反馈
应用商店评论持续提供真实用户反馈，有助于改进软件需求。然而这些评论往往杂乱、非正式，难以人工大规模分析。尽管已有自动化方法，但多数复现效果差，难以产出可直接用于敏捷项目的需求待办列表。本研究评估了GPT-3.5 Turbo、Gemini 2.0 Flash和Mistral 7B Instruct等大语言模型，能否直接从原始应用评论生成可用的用户故事。基于包含1000+条健康类应用评论的Mini-BAR数据集，采用零样本、单样本和双样本提示方法进行测试。通过RUST框架的人工评估及在UStAI上微调的RoBERTa分类器，评估生成故事的整体质量。结果表明，使用少样本提示时，大模型生成的故事在流畅性和格式规范性上可媲美甚至超越人类；但依然难以生成独立且独特的用户故事，这对其构建高质量敏捷待办清单构成挑战。总体而言，研究证明大模型能可靠地将非结构化评论转化为可操作的软件需求，为开发者提供明确方向，将用户反馈转化为实质性改进。
原文摘要 · Abstract (English)
App store reviews provide a constant flow of real user feedback that can help improve software requirements. However, these reviews are often messy, informal, and difficult to analyze manually at scale. Although automated techniques exist, many do not perform well when replicated and often fail to produce clean, backlog-ready user stories for agile projects. In this study, we evaluate how well large language models (LLMs) such as GPT-3.5 Turbo, Gemini 2.0 Flash, and Mistral 7B Instruct can generate usable user stories directly from raw app reviews. Using the Mini-BAR dataset of 1,000+ health app reviews, we tested zero-shot, one-shot, and two-shot prompting methods. We evaluated the generated user stories using both human judgment (via the RUST framework) and a RoBERTa classifier fine-tuned on UStAI to assess their overall quality. Our results show that LLMs can match or even outperform humans in writing fluent, well-formatted user stories, especially when few-shot prompts are used. However, they still struggle to produce independent and unique user stories, which are essential for building a strong agile backlog. Overall, our findings show how LLMs can reliably turn unstructured app reviews into actionable software requirements, providing developers with clear guidance to turn user feedback into meaningful improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。