通过生成反事实样本提升语法纠错模型在变化上下文中的稳定性
CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correction
- 构建反事实生成框架，改变上下文但保留错误模式和句法结构
- 在扰动数据集上实现最高20.8点的F0.5提升，显著增强模型鲁棒性
- 适合需要高鲁棒性的自然语言处理场景，如教育评测与自动校对
语法纠错（GEC）系统通常在基准数据集上训练和评估，但当上下文发生微小扰动或扩展时，性能会急剧下降，表明现有模型难以理解不同上下文中错误模式的变化。本文深入研究了GEC任务中的反事实问题，即上下文细微变化可能导致标签翻转。为此，我们提出CoCoGEC，一种反事实生成框架，通过修改训练样本中与错误无关的上下文来生成反事实实例。该框架系统地生成句子内和句子间反事实样本，保持原始实例的错误模式与句法结构；并通过选择标签翻转且具有高GEC互信息系数的样本进行修正。大量实验表明，该方法显著提升了GEC模型的稳定性，优于多种数据增强基线。尤其在扰动后的BEA-19*、CoNLL-14*和TEM-8*数据集上，分别获得+9.9、+11.3和+20.8的绝对F0.5提升。代码已开源：https://github.com/Quinnok/CoCoGEC
原文摘要 · Abstract (English)
Grammatical error correction (GEC) systems are usually trained and evaluated on GEC benchmarks, but their performance often drops sharply once the surrounding context is slightly perturbed or extended. This indicates that the existing GEC models usually fail to understand the error patterns in the varying contexts. In this paper, we thoroughly investigate the counterfactuals for GEC tasks, where the subtle changes to the contexts could lead to the label flipping issue. We propose CoCoGEC, a counterfactual generation framework that creates copies of training instances with error-irrelevant contexts altered. Our framework systematically generates counterfactuals by (1) generating intra- and inter-sentence counterfactuals that maintain the error patterns as well as syntax of the original instances by altering the word-level and sentence-level contexts; (2) revising the generated counterfactuals by selecting the instances with flipped labels and high GEC Mutual Information (MI) coefficient. Extensive experiments show that our method substantially improves the stability of GEC models, outperforming a set of data augmentation baselines. Particularly, it could achieve absolute F0.5 gains of +9.9, +11.3, and +20.8 points on the perturbed BEA-19*,CoNLL-14*, and TEM-8* data set.Our code is released at https://github.com/Quinnok/CoCoGEC
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。