自动批改物理试卷，用代数重写系统验证学生答案正确性
The AlphaPhysics Term Rewriting System for Marking Algebraic Expressions in Physics Exams
- 用大模型清理错误，将学生答案转为机器可读形式
- 在1500+真实考题上测试，准确率高且能处理三角函数表达式
- 专为物理题设计的代数重写系统，确保推理过程可靠
我们提出一种自动批改物理考试的方法。该任务需判断学生输入的答案是否与标准解一致，具有挑战性。我们结合计算机代数系统、SMT求解器和专为物理问题设计的术语重写系统来解决。使用大语言模型对学生的答案进行语义理解并修正错误，将其转化为机器可读格式。形式化后，采用自动化推理技术评估答案正确性。我们对比了两种定理证明方法：现成的SMT求解和针对含三角函数的物理问题优化的术语重写系统。术语重写系统的构建及终止性、收敛性证明较为复杂，文中详述。系统在2023年澳大利亚物理奥林匹克竞赛的超过1500份真实学生答卷上进行了评估。
原文摘要 · Abstract (English)
We present our method for automatically marking Physics exams. The marking problem consists in assessing typed student answers for correctness with respect to a ground truth solution. This is a challenging problem that we seek to tackle using a combination of a computer algebra system, an SMT solver and a term rewriting system. A Large Language Model is used to interpret and remove errors from student responses and rewrite these in a machine readable format. Once formalized and language-aligned, the next step then consists in applying automated reasoning techniques for assessing student solution correctness. We consider two methods of automated theorem proving: off-the-shelf SMT solving and term rewriting systems tailored for physics problems involving trigonometric expressions. The development of the term rewrite system and establishing termination and confluence properties was not trivial, and we describe it in some detail in the paper. We evaluate our system on a rich pool of over 1500 real-world student exam responses from the 2023 Australian Physics Olympiad.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。