对比聊天与结构化反馈，发现后者更利于数学证明学习。
Chat-Based Support Alone May Not Be Enough: Comparing Conversational and Embedded LLM Feedback for Mathematical Proof Learning
- 用结构化反馈工具提供写作式证明的即时批改。
- 聊天机器人使用多的学生中期成绩反而下降。
- 自信心弱或基础差的学生更依赖系统，但效果有限。
我们评估了GPTutor，一个为本科生离散数学课程设计的基于大模型的辅导系统。该系统整合了两个由大模型支持的工具：针对学生书面证明尝试的结构化证明评审工具，以及用于数学问题咨询的聊天机器人。在148名学生的分阶段访问实验中，早期使用系统的组在仅可使用系统期间的作业表现更高，但这一优势未体现在期末考试成绩上。使用日志显示，自信心较低或前期考试成绩较差的学生更频繁地使用两个组件。通过人工标注和自动化分类器生成的会话级行为标签，揭示了学生如何与聊天机器人互动（如寻求答案或求助）。在控制前期成绩和自我效能感后，聊天机器人使用频率及寻求答案行为与后续中期成绩呈负相关，而证明评审工具的使用则无明显独立关联。结果表明，仅靠聊天机器人支持可能无法有效促进数学证明学习向独立评估的迁移，而以任务为中心的结构化反馈则影响较小。
原文摘要 · Abstract (English)
We evaluate GPTutor, an LLM-powered tutoring system for an undergraduate discrete mathematics course. It integrates two LLM-supported tools: a structured proof-review tool that provides embedded feedback on students' written proof attempts, and a chatbot for math questions. In a staggered-access study with 148 students, earlier access was associated with higher homework performance during the interval when only the experimental group could use the system, while we did not observe this performance increase transfer to exam scores. Usage logs show that students with lower self-efficacy and prior exam performance used both components more frequently. Session-level behavioral labels, produced by human coding and scaled using an automated classifier, characterize how students engaged with the chatbot (e.g., answer-seeking or help-seeking). In models controlling for prior performance and self-efficacy, higher chatbot usage and answer-seeking behavior were negatively associated with subsequent midterm performance, whereas proof-review usage showed no detectable independent association. Together, the findings suggest that chatbot-based support alone may not reliably support transfer to independent assessment of math proof-learning outcomes, whereas work-anchored, structured feedback appears less associated with reduced learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。