SE研究者用ML时数据与评估难题多,教学方法需改进。
Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education
- 从研究、教学、评审三方面调研SE学者用ML实践
- 超参数调优等好做法少于20%论文采用,评估常忽视非功能属性
- 适合教育者和审稿人参考,推动更全面的ML应用规范
背景:机器学习(ML)对软件工程(SE)影响深远,但现有研究多关注从业者,忽略研究者在科研、教学与评审中应用ML的实践。目的:从SE研究者视角,揭示其在研究、教学及评审中应用ML的实践与挑战。方法:分析熟悉ML的SE研究者及其发表的论文,采用扎根理论编码与定性分析,考察数据收集、模型训练、评估实践,以及评审与教育中的观点。结果:实践中存在多样化做法,但超参数调优等推荐方法在不足20%的文献中出现;常见挑战包括数据处理、模型评估(含非功能属性)、以及融入人类专家判断;教学中虽普遍开展实践操作,传统讲授仍占主导。结论:尽管将ML应用于SE已成共识,但关键实践仍存显著缺口。通过完善指南、推广多元教学法、重视被忽视的实践环节,可助力该领域发展。
原文摘要 · Abstract (English)
Context: Machine Learning (ML) significantly impacts Software Engineering (SE), but studies mainly focus on practitioners, neglecting researchers. This overlooks practices and challenges in teaching, researching, or reviewing ML applications in SE. Objective: This study aims to contribute to the knowledge, about the synergy between ML and SE from the perspective of SE researchers, by providing insights into the practices followed when researching, teaching, and reviewing SE studies that apply ML. Method: We analyzed SE researchers familiar with ML or who authored SE articles using ML, along with the articles themselves. We examined practices, SE tasks addressed with ML, challenges faced, and reviewers' and educators' perspectives using grounded theory coding and qualitative analysis. Results: We found diverse practices focusing on data collection, model training, and evaluation. Some recommended practices (e.g., hyperparameter tuning) appeared in less than 20\% of literature. Common challenges involve data handling, model evaluation (incl. non-functional properties), and involving human expertise in evaluation. Hands-on activities are common in education, though traditional methods persist. Conclusion: Despite accepted practices in applying ML to SE, significant gaps remain. By enhancing guidelines, adopting diverse teaching methods, and emphasizing underrepresented practices, the SE community can bridge these gaps and advance the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。