用大模型分析留学生反思文本,一次比较多个身份变量的情感差异。
LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments
- 用LLM自动分析151份留学反思,量化不同身份变量的情感差异。
- 发现只有海外生活经历影响学生对语言表达的情感评价。
- 适合想高效对比多组人群的教育研究者使用。
书面反思作业为学生提供了批判性自我评估、意义建构和学习过程的机会,同时也为质性教育研究提供了丰富数据。然而,质性数据分析耗时较长,尤其在跨群体比较时,通常仅限于单一变量(如性别二分)。大语言模型(LLMs)近期被探索作为质性研究助手。本研究以一项留学项目中151名本科生的反思文本为纵向案例,探究LLM辅助情感分析如何实现计算与主题分析相结合的混合方法研究。首先,通过统计检验定量比较七个不同学生身份/生活经验变量下的情感差异;随后,基于结果开展质性分析,探讨差异背后的原因。研究发现,在留学生群体中,仅有海外生活经历显著影响其对语言与沟通行为的情感评价。该流程为质性研究者更便捷地探查多变量群体差异提供了新路径。
原文摘要 · Abstract (English)
Written reflection assignments give students valuable opportunities for critical self-assessment, meaning making, and learning processing. Additionally, such reflections provide rich data for qualitative education research. However, qualitative data can be time-consuming to analyze. It is even more time-intensive to qualitatively compare findings between different groups of participants, usually limiting comparison to, at most, one variable (e.g., binary gender). Large language models (LLMs) have recently begun to be critically evaluated for use as qualitative research assistants. Using a longitudinal case of written student reflections (n=151) from a study abroad program, we investigate how LLM-assisted sentiment analysis can enable longitudinal mixed-methods research combining computational and thematic analyses. First, statistical testing is used to quantitatively compare sentiment differences according to seven different student identity/lived experience variables. Then, these results inform qualitative data analysis to investigate the reasons underlying these differences. For the case of undergraduate students studying abroad, we found that prior experience living abroad was the only personal variable impacting students' sentiments of their verbal language and communication behaviors. This workflow has implications for how qualitative researchers can more easily probe multiple variables when comparing participants from different demographic groups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。