用清洗后的Lang-8数据微调BERT，GED准确率超98%。
Enhancing Grammatical Error Detection using BERT with Cleaned Lang-8 Dataset
- 用自研清洗版Lang-8数据微调BERT模型
- 测试集F1达90.53%，训练集准确率98.49%
- 证明数据清洗比模型更大更有效
本文提出一种基于大语言模型的语法错误检测（GED）改进方法，该任务在诸多应用中极具挑战性且至关重要。传统方法依赖人工设计特征，近年神经网络已能自动提取特征，显著提升性能。规则系统F1为0.50–0.60，早期机器学习模型达0.65–0.75，而此前深度学习模型如Bi-LSTM的F1在0.80–0.90之间。本研究使用我们严格清洗的Lang8数据集，对多种Transformer模型进行微调。实验显示，BERT-base-uncased在训练数据上取得98.49%准确率和0.91 F1，测试数据上达90.53%准确率。增大模型至BERT-large-uncased或RoBERTa-large未带来明显提升，表明更大模型并非必然更优。结果表明，严谨的数据清洗与简洁的Transformer模型足以显著提升GED质量。
原文摘要 · Abstract (English)
This paper presents an improved LLM based model for Grammatical Error Detection (GED), which is a very challenging and equally important problem for many applications. The traditional approach to GED involved hand-designed features, but recently, Neural Networks (NN) have automated the discovery of these features, improving performance in GED. Traditional rule-based systems have an F1 score of 0.50-0.60 and earlier machine learning models give an F1 score of 0.65-0.75, including decision trees and simple neural networks. Previous deep learning models, for example, Bi-LSTM, have reported F1 scores within the range from 0.80 to 0.90. In our study, we have fine-tuned various transformer models using the Lang8 dataset rigorously cleaned by us. In our experiments, the BERT-base-uncased model gave an impressive performance with an F1 score of 0.91 and accuracy of 98.49% on training data and 90.53% on testing data, also showcasing the importance of data cleaning. Increasing model size using BERT-large-uncased or RoBERTa-large did not give any noticeable improvements in performance or advantage for this task, underscoring that larger models are not always better. Our results clearly show how far rigorous data cleaning and simple transformer-based models can go toward significantly improving the quality of GED.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。