对比GPT三版本在统计考试中的表现，发现免费版差距明显。
Generative AI Takes a Statistics Exam: A Comparison of Performance between ChatGPT3.5, ChatGPT4, and ChatGPT4o-mini
- 用16道研究生统计题测试GPT3.5、GPT4和GPT4o-mini
- GPT3.5不及格，GPT4优秀，GPT4o-mini居中
- 免费版与付费版差异大，适合关注教育公平的读者
许多人认为生成式AI作为私人导师有潜力缩小资源丰富与资源匮乏学校之间的学习差距。这一目标成立的前提是付费版与免费版平台在准确率上相当。本实验使用同一份包含16道题的统计学考试，评估GPT 3.5、GPT 4.0和GPT4o-mini的表现，该考试面向一年级研究生。尽管不提倡用AI完成考试，但通过真实考题可分析ChatGPT对典型统计问题的回答特征。结果显示，GPT 3.5未能通过考试，GPT4表现良好，而GPT4o-mini介于两者之间。我们进一步采用文本分析方法（如阅读水平评估和主题建模）比较各版本回答差异。结果表明，GPT3.5与GPT4o-mini在回答特征上更相似，均不同于GPT4。本研究仅聚焦ChatGPT，因其目前在高校中最广泛使用。
原文摘要 · Abstract (English)
Many believe that use of generative AI as a private tutor has the potential to shrink access and achievement gaps between students and schools with abundant resources versus those with fewer resources. Shrinking the gap is possible only if paid and free versions of the platforms perform with the same accuracy. In this experiment, we investigate the performance of GPT versions 3.5, 4.0, and 4o-mini on the same 16-question statistics exam given to a class of first-year graduate students. While we do not advocate using any generative AI platform to complete an exam, the use of exam questions allows us to explore aspects of ChatGPT's responses to typical questions that students might encounter in a statistics course. Results on accuracy indicate that GPT 3.5 would fail the exam, GPT4 would perform well, and GPT4o-mini would perform somewhere in between. While we acknowledge the existence of other Generative AI/LLMs, our discussion concerns only ChatGPT because it is the most widely used platform on college campuses at this time. We further investigate differences among the AI platforms in the answers for each problem using methods developed for text analytics, such as reading level evaluation and topic modeling. Results indicate that GPT3.5 and 4o-mini have characteristics that are more similar than either of them have with GPT4.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。