用大模型分析评论情感，精准衡量在线学习参与度。
LLM-SEM: A Sentiment-Based Student Engagement Metric Using LLMS for E-Learning Platforms
- 结合视频数据与评论情感分析，生成多维度参与度指标。
- 在多个课程层级上实现高精度量化，支持大规模应用。
- 适合教育科技平台优化教学设计与学生互动策略。
当前在线学习平台的学生参与度分析方法，如自动化系统和传统问卷，普遍存在文本情感模糊、元数据有限、样本量小及可扩展性差等问题。本文提出基于大语言模型的参与度度量方法 LLM-SEM，融合视频元数据与学生评论的情感分析，通过近期大型语言模型（LLMs）生成高质量情感预测，缓解文本模糊性，并对观看数、点赞数等关键特征进行归一化处理。该方法综合全面元数据与情感极性得分，在课程与课时层面实现参与度评估。我们采用人工标注的情感数据集对 TXLM-RoBERTa 进行微调以提升预测准确率，并使用 Ollama 提供的 LLama 3B 和 Gemma 9B 模型开展实验，验证了 LLM-SEM 在可扩展性与准确性方面的有效性。
原文摘要 · Abstract (English)
Current methods for analyzing student engagement in e-learning platforms, including automated systems, often struggle with challenges such as handling fuzzy sentiment in text comments and relying on limited metadata. Traditional approaches, such as surveys and questionnaires, also face issues like small sample sizes and scalability. In this paper, we introduce LLM-SEM (Language Model-Based Student Engagement Metric), a novel approach that leverages video metadata and sentiment analysis of student comments to measure engagement. By utilizing recent Large Language Models (LLMs), we generate high-quality sentiment predictions to mitigate text fuzziness and normalize key features such as views and likes. Our holistic method combines comprehensive metadata with sentiment polarity scores to gauge engagement at both the course and lesson levels. Extensive experiments were conducted to evaluate various LLM models, demonstrating the effectiveness of LLM-SEM in providing a scalable and accurate measure of student engagement. We fine-tuned TXLM-RoBERTa using human-annotated sentiment datasets to enhance prediction accuracy and utilized LLama 3B, and Gemma 9B from Ollama.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。