用大模型模拟学生修改，反向优化写作反馈生成
Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions
- 通过大模型模拟学生修改来迭代优化反馈生成
- 在经济作文任务中显著提升学生修改质量
- 无需显式训练即可增强教学价值，适合教育AI研究者
提供反馈被广泛认为对提升学生写作能力至关重要。近年来，语言模型（LMs）的发展使得自动生成可操作且与人类指定属性一致的反馈成为可能。然而，这些模型生成的反馈是否真正有效提升学生修改质量仍不明确。此外，由于缺乏关于哪些具体属性能促进修改表现的共识，精准提示大模型生成反馈也颇具挑战。为此，我们提出PROF——通过学习大模型模拟的学生修改来生成反馈的方法。PROF通过直接最大化由大模型模拟的学生整体修改表现，迭代优化反馈生成器。在经济作文任务上的实证测试表明，该方法不仅在提升学生写作质量方面优于多种基线方法，还展现出更强的教学价值，尽管其未显式针对此目标进行训练。
原文摘要 · Abstract (English)
Providing feedback is widely recognized as crucial for refining students' writing skills. Recent advances in language models (LMs) have made it possible to automatically generate feedback that is actionable and well-aligned with human-specified attributes. However, it remains unclear whether the feedback generated by these models is truly effective in enhancing the quality of student revisions. Moreover, prompting LMs with a precise set of instructions to generate feedback is nontrivial due to the lack of consensus regarding the specific attributes that can lead to improved revising performance. To address these challenges, we propose PROF that PROduces Feedback via learning from LM simulated student revisions. PROF aims to iteratively optimize the feedback generator by directly maximizing the effectiveness of students' overall revising performance as simulated by LMs. Focusing on an economic essay assignment, we empirically test the efficacy of PROF and observe that our approach not only surpasses a variety of baseline methods in effectiveness of improving students' writing but also demonstrates enhanced pedagogical values, even though it was not explicitly trained for this aspect.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。