用大模型预测代码任务所需技能，帮新人选对开源项目贡献目标
SkillScope: A Tool to Predict Fine-Grained Skills Needed to Solve Issues on GitHub
- 结合大模型与随机森林，预测解决开源问题所需的多层级编程技能
- 在Java项目上实现91%精度、88%召回率，平均F1达89%
- 适合新手找合适任务，也适用于项目维护者合理分配工作
新贡献者在加入新的开源软件（OSS）项目时，常因无法判断任务所需知识而难以找到可胜任的工作。问题追踪系统缺乏对完成任务所需技能的详细说明，而这些说明本身又难生成。已有研究尝试通过标注问题类型、难度和技能标签来缓解此问题，但现有标签集有限，语义细节不足，难以有效引导贡献者。为此，本文探索使用大语言模型（LLMs）与随机森林（RF）方法，预测解决开放问题所需的多层次技能。我们提出一个名为SkillScope的新工具，从托管在GitHub上的Java项目中检索问题，并预测解决它们所需的多层级编程技能。案例研究表明，SkillScope能为任务预测出217个多层次技能，平均精度达91%，召回率为88%，F-measure为89%。实践者可借助该工具更精准地分配或选择任务。
原文摘要 · Abstract (English)
New contributors often struggle to find tasks that they can tackle when onboarding onto a new Open Source Software (OSS) project. One reason for this difficulty is that issue trackers lack explanations about the knowledge or skills needed to complete a given task successfully. These explanations can be complex and time-consuming to produce. Past research has partially addressed this problem by labeling issues with issue types, issue difficulty level, and issue skills. However, current approaches are limited to a small set of labels and lack in-depth details about their semantics, which may not sufficiently help contributors identify suitable issues. To surmount this limitation, this paper explores large language models (LLMs) and Random Forest (RF) to predict the multilevel skills required to solve the open issues. We introduce a novel tool, SkillScope, which retrieves current issues from Java projects hosted on GitHub and predicts the multilevel programming skills required to resolve these issues. In a case study, we demonstrate that SkillScope could predict 217 multilevel skills for tasks with 91% precision, 88% recall, and 89% F-measure on average. Practitioners can use this tool to better delegate or choose tasks to solve in OSS projects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。