用小模型舰队从目标生成用户故事,提升需求获取效率与隐私保护。
Goal2Story: A Multi-Agent Fleet based on Privately Enabled sLLMs for Impacting Mapping on Requirements Elicitation
- 基于影响映射框架,用私有部署小模型构建多智能体系统。
- 在1000+用户故事数据集上表现优于大模型基线,关键指标提升显著。
- 适合敏捷开发中需保护数据隐私且资源受限的团队使用。
随着快速迭代导致需求不断漂移,敏捷开发成为主流范式。目标驱动的需求获取(RE)是敏捷开发中的关键但极具挑战的任务,因其高度依赖自适应规划与高效协作。近年来,AI代理在需求分析中展现出显著潜力,可大幅节省利益相关者的时间与精力。然而,现有研究主要集中于功能型需求获取,尚未有工作实现从目标到用户故事的完整链路衔接。此外,考虑到大模型服务的成本及对数据与创意的保护需求,应更充分地利用私有部署的小规模语言模型(sLLMs)于需求获取。为此,我们提出Goal2Story:一个基于影响映射(IM)框架、仅使用低成本sLLMs的多智能体舰队,用于目标驱动的需求获取。同时,我们构建了StorySeek数据集,包含超过1,000个带有目标和项目上下文信息的用户故事,并提出半自动构建方法。评估方面,我们设计了两个指标:事实一致性命中率(FHR)衡量生成故事与数据集的一致性,质量与一致性评估(QuACE)用于评价生成质量。实验结果表明,Goal2Story在性能上超越采用强大LLM的Super-Agent基线,且通过思维链(CoT)与智能体角色配置带来关键指标提升,还展现出识别潜在需求的能力。
原文摘要 · Abstract (English)
As requirements drift with rapid iterations, agile development becomes the dominant paradigm. Goal-driven Requirements Elicitation (RE) is a pivotal yet challenging task in agile project development due to its heavy tangling with adaptive planning and efficient collaboration. Recently, AI agents have shown promising ability in supporting requirements analysis by saving significant time and effort for stakeholders. However, current research mainly focuses on functional RE, and research works have not been reported bridging the long journey from goal to user stories. Moreover, considering the cost of LLM facilities and the need for data and idea protection, privately hosted small-sized LLM should be further utilized in RE. To address these challenges, we propose Goal2Story, a multi-agent fleet that adopts the Impact Mapping (IM) framework while merely using cost-effective sLLMs for goal-driven RE. Moreover, we introduce a StorySeek dataset that contains over 1,000 user stories (USs) with corresponding goals and project context information, as well as the semi-automatic dataset construction method. For evaluation, we proposed two metrics: Factuality Hit Rate (FHR) to measure consistency between the generated USs with the dataset and Quality And Consistency Evaluation (QuACE) to evaluate the quality of the generated USs. Experimental results demonstrate that Goal2Story outperforms the baseline performance of the Super-Agent adopting powerful LLMs, while also showcasing the performance improvements in key metrics brought by CoT and Agent Profile to Goal2Story, as well as its exploration in identifying latent needs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。