为英语学习者写作错误设计智能反馈框架,提升纠错的教育价值。
Annotating Errors in English Learners' Written Language Production: Advancing Automated Written Feedback Systems
- 构建错误类型与可迁移性标注体系,定位学习者知识盲区。
- 采集带人工反馈标签的数据集,验证生成式反馈的有效性。
- 适合语言教育研究者与自动批改系统开发者参考。
自然语言处理的进展推动了自动化写作评估(AWE)系统的发展,能有效修正语法错误。然而,现有系统多侧重直接修改,缺乏对语言学习需求的针对性,常忽略错误背后的学习机制。针对此问题,本文提出一种标注框架,通过分析错误类型和规则可迁移性,揭示学习者的知识缺口。基于该框架,构建了一个包含标注错误及其对应人工反馈评论的数据集,每条反馈标记为直接修正或提示。利用大规模语言模型,比较关键词引导、无关键词及模板引导三类反馈生成方法,并由母语教师从相关性、事实性和可理解性角度评估输出质量。研究报告了数据集构建过程及各系统性能对比结果。
原文摘要 · Abstract (English)
Recent advances in natural language processing (NLP) have contributed to the development of automated writing evaluation (AWE) systems that can correct grammatical errors. However, while these systems are effective at improving text, they are not optimally designed for language learning. They favor direct revisions, often with a click-to-fix functionality that can be applied without considering the reason for the correction. Meanwhile, depending on the error type, learners may benefit most from simple explanations and strategically indirect hints, especially on generalizable grammatical rules. To support the generation of such feedback, we introduce an annotation framework that models each error's error type and generalizability. For error type classification, we introduce a typology focused on inferring learners' knowledge gaps by connecting their errors to specific grammatical patterns. Following this framework, we collect a dataset of annotated learner errors and corresponding human-written feedback comments, each labeled as a direct correction or hint. With this data, we evaluate keyword-guided, keyword-free, and template-guided methods of generating feedback using large language models (LLMs). Human teachers examined each system's outputs, assessing them on grounds including relevance, factuality, and comprehensibility. We report on the development of the dataset and the comparative performance of the systems investigated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。