分析1.7万条中小学生生成式AI对话,按主题和任务双维度分类。
Thematic and Task-Based Categorization of K-12 GenAI Usages with Hierarchical Topic Modeling
- 用改进的层级主题模型,从内容和任务双视角分类学生对话。
- 发现大量新应用场景,如跨学科写作与解释性学习。
- 适合教育研究者、教师及关注AI教学应用的人群。
我们分析了跨越数月、多所学校和学科的匿名课堂互动数据,采用一种新颖且简单的主题建模方法,对超过17,000条由学生、教师及ChatGPT生成的消息进行双重分类:内容(如自然、人物)和任务(如写作、解释)。通过分别对每个维度进行层级分类,并附带典型提示示例,提供了宏观概览与具体洞察。现有研究大多缺乏内容或主题分类,尽管任务分类在教育领域较常见,但多数未基于真实的K-12数据。因此,我们的分析揭示了若干新颖的应用场景。在推导这些洞见时,我们发现许多经典和新兴的计算方法(如主题建模)在处理大规模文本时表现不佳,因此直接使用经过充分预处理的状态先进大语言模型(LLMs),通过明确指令获得更符合人类认知的层级主题结构。研究结果有助于研究人员、教师和学生丰富生成式AI在教育中的应用,同时讨论也指出了未来研究需关注的若干问题与挑战。
原文摘要 · Abstract (English)
We analyze anonymous interaction data of minors in class-rooms spanning several months, schools, and subjects employing a novel, simple topic modeling approach. Specifically, we categorize more than 17,000 messages generated by students, teachers, and ChatGPT in two dimensions: content (such as nature and people) and tasks (such as writing and explaining). Our hierarchical categorization done separately for each dimension includes exemplary prompts, and provides both a high-level overview as well as tangible insights. Prior works mostly lack a content or thematic categorization. While task categorizations are more prevalent in education, most have not been supported by real-world data for K-12. In turn, it is not surprising that our analysis yielded a number of novel applications. In deriving these insights, we found that many of the well-established classical and emerging computational methods, i.e., topic modeling, for analysis of large amounts of texts underperform, leading us to directly apply state-of-the-art LLMs with adequate pre-processing to achieve hierarchical topic structures with better human alignment through explicit instructions than prior approaches. Our findings support fellow researchers, teachers and students in enriching the usage of GenAI, while our discussion also highlights a number of concerns and open questions for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。