让模型通过题目文本理解学生能力,更好预测新题表现。
Just Read the Question: Enabling Generalization to New Assessment Items with Text Awareness
- 用题目文本嵌入增强模型,提升对新题的适应能力。
- 在已见题上表现持平,在未见题上显著优于原有方法。
- 适合需要快速适配新试题的智能教育系统使用。
机器学习被用于教育评估,以实现对学生表现和题目间学习关系的细粒度预测。但多数方法在引入新题目时受限于历史数据依赖。本文提出Text-LENS,通过扩展基于部分变分自编码器的LENS模型,引入题目文本嵌入,探索其对预测性能及未见题泛化能力的影响。在两个数据集上测试:Eedi(含题目内容的公开数据集)和LLM-Sim(由大语言模型生成的新题数据集)。结果表明,Text-LENS在已见题目上表现与LENS相当,在涉及未见题的各种条件下均有提升,能有效从题目文本中学习学生能力并准确预测其在新题上的表现。
原文摘要 · Abstract (English)
Machine learning has been proposed as a way to improve educational assessment by making fine-grained predictions about student performance and learning relationships between items. One challenge with many machine learning approaches is incorporating new items, as these approaches rely heavily on historical data. We develop Text-LENS by extending the LENS partial variational auto-encoder for educational assessment to leverage item text embeddings, and explore the impact on predictive performance and generalization to previously unseen items. We examine performance on two datasets: Eedi, a publicly available dataset that includes item content, and LLM-Sim, a novel dataset with test items produced by an LLM. We find that Text-LENS matches LENS' performance on seen items and improves upon it in a variety of conditions involving unseen items; it effectively learns student proficiency from and makes predictions about student performance on new items.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。