arXiv:2510.19858cs.CL2025-10

用自动化方法分析在线学习中知识构建,支持大规模教育数据洞察。

Analysing Knowledge Construction in Online Learning: Adapting the Interaction Analysis Model for Unstructured Large-Scale Discourse

  • 基于互动分析模型设计代码本,划分四类知识建构行为
  • DeBERTa-v3-large模型在2万条评论上实现0.841的宏平均F1分数
  • 适用于医学、编程等结构化强的领域,可为在线教学设计提供依据

在线课程与社交媒体的快速发展产生了大量非结构化学习者生成文本。理解学习者在这些空间中如何构建知识,对分析学习过程、优化内容设计和实现规模化反馈至关重要。然而,现有方法多依赖人工编码结构化讨论区,难以应对在线学习中碎片化的对话。本研究提出并验证了一种结合理论驱动代码本与自动化分类器的框架,用于大规模分析非结构化在线话语中的知识建构。将评论级别知识建构分为四类:无知识建构、分享、探索与整合。三位标注员对来自YouTube教育频道的2万条评论进行编码,代码本在主数据集上表现出良好一致性(Cohen's kappa = 0.79),在四个附加教育领域中kappa值达0.85–0.93。对比了词袋基线与基于Transformer的语言模型,采用10折交叉验证,DeBERTa-v3-large模型取得最高宏平均F1分数(0.841),优于所有基线及其他Transformer模型。外部验证在四个领域中宏F1均超过0.705,迁移能力在医学与编程领域较强(话语更结构化、任务导向),而在语言与音乐领域较弱(评论更多样、依赖语境)。总体表明,基于理论的半自动化知识建构分析在大规模场景下可行,有助于将知识建构指标融入学习分析,并指导在线学习环境设计。

原文摘要 · Abstract (English)

The rapid expansion of online courses and social media has generated large volumes of unstructured learner-generated text. Understanding how learners construct knowledge in these spaces is crucial for analysing learning processes, informing content design, and providing feedback at scale. However, existing approaches typically rely on manual coding of well-structured discussion forums, which does not scale to the fragmented discourse found in online learning. This study proposes and validates a framework that combines a codebook inspired by the Interaction Analysis Model with an automated classifier to enable large-scale analysis of knowledge construction in unstructured online discourse. We adapt four comment-level categories of knowledge construction: Non-Knowledge Construction, Share, Explore, and Integrate. Three trained annotators coded a balanced sample of 20,000 comments from YouTube education channels. The codebook demonstrated strong reliability, with Cohen's kappa = 0.79 on the main dataset and 0.85--0.93 across four additional educational domains. For automated classification, bag-of-words baselines were compared with transformer-based language models using 10-fold cross-validation. A DeBERTa-v3-large model achieved the highest macro-averaged F1 score (0.841), outperforming all baselines and other transformer models. External validation on four domains yielded macro-F1 above 0.705, with stronger transfer in medicine and programming, where discourse was more structured and task-focused, and weaker transfer in language and music, where comments were more varied and context-dependent. Overall, the study shows that theory-driven, semi-automated analysis of knowledge construction at scale is feasible, enabling the integration of knowledge-construction indicators into learning analytics and the design of online learning environments.

知识建构学习分析NLP应用教育科技

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。