将语法纠错拆解为7个子任务,用多头模型提升准确率。
Multi-head Sequence Tagging Model for Grammatical Error Correction
- 分7类纠错任务,每类配独立分类头,统一建模
- 在BEA-19和CoNLL-14上达F0.5 77.0/69.1
- 适合需要高精度语法修正的研究与应用
为解决语法错误修正(GEC)问题,需建立源序列与目标序列间的映射,二者仅在少数片段上不同。因此,研究转向非自回归或序列标注模型,将GEC从序列到序列任务简化为对输入词元打上编辑命令标签,从庞大的编辑空间中选择。由于类别数量庞大且训练数据有限,现有序列标注方法难以覆盖广泛错误类型。为此,本文进一步将GEC分解为七个相关子任务:插入、删除、合并、替换、转换、检测与修正,以修正为主。为每个子任务设计独立分类头,提出新型多头多任务学习模型,有效利用训练数据并融合相关任务信号。为缓解数据稀缺问题,引入新去噪自编码器生成合成数据用于预训练;同时提出字符级转换机制,增强序列到编辑的映射能力,提升词汇覆盖。单模型与集成模型在BEA-19(测试集)上分别取得F0.5 74.4/77.0,在CoNLL-14(测试集)上为68.6/69.1。在JFLEG测试集上,GLEU得分分别为61.6和61.7,显著优于近期最先进方法。
原文摘要 · Abstract (English)
To solve the Grammatical Error Correction (GEC) problem , a mapping between a source sequence and a target one is needed, where the two differ only on few spans. For this reason, the attention has been shifted to the non-autoregressive or sequence tagging models. In which, the GEC has been simplified from Seq2Seq to labeling the input tokens with edit commands chosen from a large edit space. Due to this large number of classes and the limitation of the available datasets, the current sequence tagging approaches still have some issues handling a broad range of grammatical errors just by being laser-focused on one single task. To this end, we simplified the GEC further by dividing it into seven related subtasks: Insertion, Deletion, Merge, Substitution, Transformation, Detection, and Correction, with Correction being our primary focus. A distinct classification head is dedicated to each of these subtasks. the novel multi-head and multi-task learning model is proposed to effectively utilize training data and harness the information from related task training signals. To mitigate the limited number of available training samples, a new denoising autoencoder is used to generate a new synthetic dataset to be used for pretraining. Additionally, a new character-level transformation is proposed to enhance the sequence-to-edit function and improve the model's vocabulary coverage. Our single/ensemble model achieves an F0.5 of 74.4/77.0, and 68.6/69.1 on BEA-19 (test) and CoNLL-14 (test) respectively. Moreover, evaluated on JFLEG test set, the GLEU scores are 61.6 and 61.7 for the single and ensemble models, respectively. It mostly outperforms recently published state-of-the-art results by a considerable margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。