不同模型对教育变量重要性评估差异大，多模型验证更可靠。
Rashomon effect in Educational Research: Why More is Better Than One for Measuring the Importance of the Variables?
- 用多种树模型构建鲁棒性集合，提升预测准确率2-6%
- 性别、家庭背景等变量重要性在多分类任务中不一致
- 建议跨模型验证，避免单一模型误导结论
本研究探讨了在学生人口统计特征用于学业表现预测时，Rashomon效应如何影响变量重要性。研究采用决策树、随机森林、LightGBM和XGBoost等机器学习算法，在开放大学学习分析数据集上训练一系列简单而准确的模型，形成Rashomon集合。结果显示，该集合可将预测准确率提升2%-6%。变量重要性分析表明，二分类任务中的结果比多分类更一致可靠，揭示了多目标预测的复杂性。关键人口统计变量imd_band和highest_education被识别为重要，但其重要性随课程变化，尤其在课程DDD中差异显著。研究强调模型选择的影响，提醒在泛化结果时需谨慎，因不同模型可能导致截然不同的变量重要性排序。实验代码已公开于https://anonymous.4open.science/r/JEDM_paper-DE9D。
原文摘要 · Abstract (English)
This study explores how the Rashomon effect influences variable importance in the context of student demographics used for academic outcomes prediction. Our research follows the way machine learning algorithms are employed in Educational Data Mining, focusing on highlighting the so-called Rashomon effect. The study uses the Rashomon set of simple-yet-accurate models trained using decision trees, random forests, light GBM, and XGBoost algorithms with the Open University Learning Analytics Dataset. We found that the Rashomon set improves the predictive accuracy by 2-6%. Variable importance analysis revealed more consistent and reliable results for binary classification than multiclass classification, highlighting the complexity of predicting multiple outcomes. Key demographic variables imd_band and highest_education were identified as vital, but their importance varied across courses, especially in course DDD. These findings underscore the importance of model choice and the need for caution in generalizing results, as different models can lead to different variable importance rankings. The codes for reproducing the experiments are available in the repository: https://anonymous.4open.science/r/JEDM_paper-DE9D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。