arXiv:2507.14642cs.AIcs.SE2025-07被引 4

用对比判断提升故事点估算效率,减少人工标注负担。

Efficient Story Point Estimation With Comparative Learning

  • 通过对比两个需求项的难度,训练模型预测故事点
  • 在16个项目中达到0.34的斯皮尔曼相关系数
  • 比直接评分更省时、更可信,适合团队协作

故事点是无单位的项目特定工作量估算,帮助开发者规划冲刺。传统上由开发团队通过计划扑克等手动方式共同估算。机器学习可减轻负担,但需依赖项目历史数据。当前最优模型如GPT2SP和FastText-SVM仅在同项目数据上训练时表现准确。本研究提出基于对比学习的框架,用于校准项目特定的故事点预测模型。不直接为每个需求分配点数,而是让开发者比较两两需求项,判断哪个更耗时。利用这些对比判断训练模型进行预测。我们基于16个项目的23,313条人工估算数据进行实证评估。该模型在预测与真实故事点间的斯皮尔曼等级相关系数平均达0.34,与现有先进回归模型性能相当甚至更优。人因实验验证了对比判断的优势:信心更高、标注时间更短、一致性相当。总体而言,相比回归方法,该对比学习方案在性能、标注效率和数据可靠性方面更具优势。

原文摘要 · Abstract (English)

Story points are unitless, project-specific effort estimates that help developers plan their sprints. Traditionally, developers have collaboratively estimated story points using planning poker or other manual techniques. Machine learning can reduce this burden, but only with sufficient context from the historical decisions made by the project team. That is, state-of-the-art models, such as GPT2SP and FastText-SVM, only make accurate (within-project) predictions when they are trained on data from the same project. The goal of this study is to streamline story point estimation by evaluating a comparative learning-based framework for calibrating project-specific story point prediction models. Instead of assigning a specific story point value to every backlog item, developers are presented with pairs of items and asked to indicate which item requires more effort. Using these comparative judgments, a machine learning model was trained to predict the story point estimates. We empirically evaluated our technique using data from 23,313 manual estimates across 16 projects. The model trained on comparative judgments achieved, on average, a 0.34 Spearman's rank correlation coefficient between its predictions and the ground truth story points. This is similar to, if not better than, the performance of a state-of-the-art regression model trained on ground truth story points. Through human subject experiments, the advantages of comparative judgments were validated - higher confidence, lower annotation time, and comparable agreement were observed for comparative judgments compared to direct ratings. In summary, the proposed comparative learning approach is more efficient than regression-based approaches, given its better performance, lower required annotation time, and higher training data reliability.

需求估算对比学习敏捷开发机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。