arXiv:2509.13359cs.CYcs.AI2025-09

测试AI在开放考试中解数学题能力,发现其表现堪比顶尖本科生。

Evaluating undergraduate mathematics examinations in the era of generative AI: a curriculum-level case study

  • 用AI模拟学生答题,评估其在无监考下的数学解题能力。
  • AI平均成绩达一等学位水平,跨课程表现高度一致。
  • 适合关注教育评估改革与AI影响的教师和课程设计者。

生成式人工智能(GenAI)工具如OpenAI的ChatGPT正在重塑教育生态,促使重新审视传统考核方式。与此同时,高校正探索替代线下闭卷考试的方案,但对非监考环境下学术诚信与教学目标的一致性表示担忧。本研究通过实证方法,生成、转录并盲评八份本科数学考试的GenAI答卷,覆盖一所罗素集团大学第一学年全部课程内容。通过组合各题目独立的AI作答,实现了对模块及整个课程体系下AI表现的合理评估。结果显示,GenAI整体达到一等学位水平,虽在不同模块间存在差异,但跨课程表现远超真实学生在监考考试中的波动性。研究证明,当前数学考核标准在生成式AI时代亟需重构,现有评价体系的育人价值可能已显著下降。

原文摘要 · Abstract (English)

Generative artificial intelligence (GenAI) tools such as OpenAI's ChatGPT are transforming the educational landscape, prompting reconsideration of traditional assessment practices. In parallel, universities are exploring alternatives to in-person, closed-book examinations, raising concerns about academic integrity and pedagogical alignment in uninvigilated settings. This study investigates whether traditional closed-book mathematics examinations retain their pedagogical relevance when hypothetically administered in uninvigilated, open-book settings with GenAI access. Adopting an empirical approach, we generate, transcribe, and blind-mark GenAI submissions to eight undergraduate mathematics examinations at a Russell Group university, spanning the entirety of the first-year curriculum. By combining independent GenAI responses to individual questions, we enable a meaningful evaluation of GenAI performance, both at the level of modules and across the first-year curriculum. We find that GenAI attainment is at the level of a first-class degree, though current performance can vary between modules. Further, we find that GenAI performance is remarkably consistent when viewed across the entire curriculum, significantly more so than that of students in invigilated examinations. Our findings evidence the need for redesigning assessments in mathematics for unsupervised settings, and highlight the potential reduction in pedagogical value of current standards in the era of generative artificial intelligence.

AI教育数学教育考试评估生成式AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。