用大模型设计学习路径，让GEC更准更快
LLMCL-GEC: Advancing Grammatical Error Correction with LLM-Driven Curriculum Learning
- 用大模型自动判断语法错误数据难易度，分步训练
- 在CoNLL14、BEA19等测试集上显著优于基线模型
- 适合想提升语法纠错效果的NLP研究者和工程师
尽管大语言模型在特定自然语言处理任务中表现优异，但在某些领域（如语法错误纠正，GEC）仍不如专用模型。受课程学习启发，本文提出基于大模型的课程学习方法，利用大模型强大的语义理解与判别能力，评估GEC训练数据的复杂度。该方法不依赖人工设计，而是模拟专家教学顺序，从易到难逐步选择数据，迭代训练预训练的T5与LLaMA系列模型。在英语GEC多个基准测试（包括CoNLL14测试集、BEA19测试集及开发集）上的实证表明，该方法显著优于基线模型和传统课程学习策略。
原文摘要 · Abstract (English)
While large-scale language models (LLMs) have demonstrated remarkable capabilities in specific natural language processing (NLP) tasks, they may still lack proficiency compared to specialized models in certain domains, such as grammatical error correction (GEC). Drawing inspiration from the concept of curriculum learning, we have delved into refining LLMs into proficient GEC experts by devising effective curriculum learning (CL) strategies. In this paper, we introduce a novel approach, termed LLM-based curriculum learning, which capitalizes on the robust semantic comprehension and discriminative prowess inherent in LLMs to gauge the complexity of GEC training data. Unlike traditional curriculum learning techniques, our method closely mirrors human expert-designed curriculums. Leveraging the proposed LLM-based CL method, we sequentially select varying levels of curriculums ranging from easy to hard, and iteratively train and refine using the pretrianed T5 and LLaMA series models. Through rigorous testing and analysis across diverse benchmark assessments in English GEC, including the CoNLL14 test, BEA19 test, and BEA19 development sets, our approach showcases a significant performance boost over baseline models and conventional curriculum learning methodologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。