对比大模型前后学生论文，发现写作长度风格显著变化。
How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses
- 分析慕课平台前后两年学生作文，对比大模型影响
- 作文长度和写作风格明显改变，关键词出现频率上升
- 主题未变但表达方式受大模型冲击，适合教育研究者参考
2022年底ChatGPT的发布在学术界引发广泛关注与担忧。一方面，其生成接近人类水平且常可通过基础事实核查的内容被视为信息获取与辅助学习的新纪元；另一方面，也有人担心其可能导致前所未有的学术不端行为。本文通过分析一门免费大学级人工智能伦理慕课中多年的学生作文数据，考察大模型兴起对在线教育的影响。数据涵盖ChatGPT发布前后的学生作答。结果表明，ChatGPT上线后，学生作文在长度和风格上均发生显著变化，与学术出版等其他领域的观察一致。同时，与人工智能及大模型相关的关键词出现频率上升，但通过动态主题建模识别的主题或核心议题整体保持稳定。
原文摘要 · Abstract (English)
The release of ChatGPT in late 2022 caused a flurry of activity and concern in the academic and educational communities. Some see the tool's ability to generate human-like text that passes at least cursory inspections for factual accuracy ``often enough'' a golden age of information retrieval and computer-assisted learning. Some, on the other hand, worry the tool may lead to unprecedented levels of academic dishonesty and cheating. In this work, we quantify some of the effects of the emergence of Large Language Models (LLMs) on online education by analyzing a multi-year dataset of student essay responses from a free university-level MOOC on AI ethics. Our dataset includes essays submitted both before and after ChatGPT's release. We find that the launch of ChatGPT coincided with significant changes in both the length and style of student essays, mirroring observations in other contexts such as academic publishing. We also observe -- as expected based on related public discourse -- changes in prevalence of key content words related to AI and LLMs, but not necessarily the general themes or topics discussed in the student essays as identified through (dynamic) topic modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。