用大模型模拟师生互动，自动生成海量教学反馈数据。
SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agents
- 两阶段大模型协作生成学生作业与教师反馈对。
- 微调后模型在900份输出中评分超越基线与原始模型。
- 适合教育科技、AI助教研发者快速构建反馈系统。
高质量作业反馈对学生成绩至关重要，但受限于时间和预算。本文提出合成教育反馈循环（SEFL），一种无需依赖真实作业与教师反馈的合成数据框架。通过两个大语言模型（LLMs）扮演教师-学生角色，模拟作业完成与形成性反馈过程，生成19.8K组合成的作业与对应评语及改进建议。基于该数据，我们微调更小、计算效率更高的LLM，使其复现高质量目标导向反馈的关键特征。通过三名LLM评委与三名人类专家对900个输出的综合评估，验证了SEFL微调模型在反馈质量上优于未微调模型和现有基线。人类利益相关者（学生与高校教师）的定性评价与评分进一步证实其社会价值。SEFL有望变革高等教育乃至更广泛领域的反馈机制。
原文摘要 · Abstract (English)
Providing high-quality feedback on student assignments is crucial for student success, but it is heavily limited by time and budgetary constraints. In this work, we introduce Synthetic Educational Feedback Loops (SEFL), a synthetic data framework designed to generate data that resembles immediate, on-demand feedback at scale without relying on extensive, real-world student assignments and teacher feedback. To obtain this type of data, two large language models (LLMs) operate in a teacher-student role to simulate assignment completion and formative feedback, generating 19.8K synthetic pairs of student work and corresponding critiques and actionable improvements from a teacher. With this data, we fine-tune smaller, more computationally efficient LLMs on these synthetic pairs, enabling them to replicate key features of high-quality, goal-oriented feedback. Through comprehensive evaluations with three LLM judges and three human experts, across a subset of 900 outputs, we demonstrate that SEFL-tuned models outperform both their untuned counterparts and an existing baseline in terms of feedback quality. The potential for societal impact is reinforced by extensive qualitative comments and ratings from human stakeholders -- both students and higher education instructors. SEFL has the potential to transform feedback processes for higher education and beyond.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。