用大模型自动发现知识点,大幅提升建模效率
KCluster: An LLM-based Clustering Approach to Knowledge Component Discovery
- 基于大模型构建新相似度度量,识别语义相近题型
- 三组数据验证:自动建模效果优于人工设计最佳模型
- 适合教育科技团队快速构建知识点体系
教师常用知识点(KC)模型将题目映射到具体知识单元。然而,为大规模题库设计KC模型对教师而言仍是一项艰巨任务,需手动分析每道题。随着生成式AI在教育中的普及,题量激增,人工设计速度难以跟上。本文提出KCluster,一种基于大语言模型(LLM)的聚类方法,通过新构建的语义相似度度量,自动识别题目的共性模式并形成知识单元。在三个数据集上验证显示,该方法能以极低人力投入生成有效KC模型,其对学生表现的预测能力超过现有最优人工设计模型。未来工作可借助此方法揭示难点知识点,并优化教学设计。
原文摘要 · Abstract (English)
Educators evaluate student knowledge using knowledge component (KC) models that map assessment questions to KCs. Still, designing KC models for large question banks remains an insurmountable challenge for instructors who need to analyze each question by hand. The growing use of Generative AI in education is expected only to aggravate this chronic deficiency of expert-designed KC models, as course engineers designing KCs struggle to keep up with the pace at which questions are generated. In this work, we propose KCluster, a novel KC discovery algorithm based on identifying clusters of congruent questions according to a new similarity metric induced by a large language model (LLM). We demonstrate in three datasets that an LLM can create an effective metric of question similarity, which a clustering algorithm can use to create KC models from questions with minimal human effort. Combining the strengths of LLM and clustering, KCluster generates descriptive KC labels and discovers KC models that predict student performance better than the best expert-designed models available. In anticipation of future work, we illustrate how KCluster can reveal insights into difficult KCs and suggest improvements to instruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。