基于认知理论的自动评估工具，可精准衡量课件多模态知识获取效果。
LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning
- 依据认知学习理论设计四维度评分体系
- 在2000+课件上训练，表现优于现有评估方法
- 适合教育科技研发与课程质量优化使用
评估基于幻灯片的多媒体教学内容质量颇具挑战。现有方法如人工评分、参考基准指标及大语言模型评估器在可扩展性、上下文捕捉或偏见控制方面存在局限。本文提出LecEval，一种基于梅耶多媒体学习认知理论的自动化评估指标，用于衡量幻灯片教学中的多模态知识获取效果。该指标采用四个维度：内容相关性（CR）、表达清晰度（EC）、逻辑结构（LS）和受众参与度（AE）。我们构建了一个包含50余门在线课程视频中超过2000张幻灯片的大规模数据集，并由人类标注员对各维度进行细粒度评分。基于该数据集训练的模型在准确性和适应性上均优于现有指标，有效缩小了自动化评估与人工判断之间的差距。相关数据集与工具包已开源：https://github.com/JoylimJY/LecEval。
原文摘要 · Abstract (English)
Evaluating the quality of slide-based multimedia instruction is challenging. Existing methods like manual assessment, reference-based metrics, and large language model evaluators face limitations in scalability, context capture, or bias. In this paper, we introduce LecEval, an automated metric grounded in Mayer's Cognitive Theory of Multimedia Learning, to evaluate multimodal knowledge acquisition in slide-based learning. LecEval assesses effectiveness using four rubrics: Content Relevance (CR), Expressive Clarity (EC), Logical Structure (LS), and Audience Engagement (AE). We curate a large-scale dataset of over 2,000 slides from more than 50 online course videos, annotated with fine-grained human ratings across these rubrics. A model trained on this dataset demonstrates superior accuracy and adaptability compared to existing metrics, bridging the gap between automated and human assessments. We release our dataset and toolkits at https://github.com/JoylimJY/LecEval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。