用嵌入模型自动评估学习资源与目标的匹配度，提升个性化教学效率。
Embedding-Based Rankings of Educational Resources based on Learning Outcome Alignment: Benchmarking, Expert Validation, and Learner Performance
- 基于文本嵌入技术构建资源与学习目标的匹配评估框架。
- 最优模型（Voyage）识别对齐率达79%，专家验证准确率达83%。
- 实证显示对齐分越高，学习表现越好，适合教育AI开发者参考。
随着在线学习发展，个性化需求日益突出。尽管教育资源不断增长，教育者仍面临选择与目标对齐且适配学习者差异的材料的挑战。大语言模型（LLMs）在生成个性化学习内容方面潜力巨大，但验证其是否覆盖预期学习成果仍需人工评审，成本高且难扩展。本文提出一种支持低成本自动评估资源与学习目标对齐性的框架。利用人工生成材料，我们对比了多种基于LLM的文本嵌入模型，发现最优模型（Voyage）在检测对齐性上达到79%准确率。随后将该模型应用于LLM生成资源，经专家评估确认其对应关系评估准确率达83%。最后，在包含360名学习者的三组实验中，对齐分数与学习表现呈显著正相关，χ²(2, N = 360) = 15.39，p < 0.001。结果表明，基于嵌入的对齐评分可实现规模化个性化，帮助教师聚焦内容适配多样性学习者需求。
原文摘要 · Abstract (English)
As the online learning landscape evolves, the need for personalization is increasingly evident. Although educational resources are burgeoning, educators face challenges selecting materials that both align with intended learning outcomes and address diverse learner needs. Large Language Models (LLMs) are attracting growing interest for their potential to create learning resources that better support personalization, but verifying coverage of intended outcomes still requires human alignment review, which is costly and limits scalability. We propose a framework that supports the cost-effective automation of evaluating alignment between educational resources and intended learning outcomes. Using human-generated materials, we benchmarked LLM-based text-embedding models and found that the most accurate model (Voyage) achieved 79% accuracy in detecting alignment. We then applied the optimal model to LLM-generated resources and, via expert evaluation, confirmed that it reliably assessed correspondence to intended outcomes (83% accuracy). Finally, in a three-group experiment with 360 learners, higher alignment scores were positively related to greater learning performance, chi-squared(2, N = 360) = 15.39, p < 0.001. These findings show that embedding-based alignment scores can facilitate scalable personalization by confirming alignment with learning outcomes, which allows teachers to focus on tailoring content to diverse learner needs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。