用强化学习优化中文纠错,减少过度修改,提升准确率。
CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards

- 三阶段框架:预训练+思维链微调+效率感知强化学习
- 在NACGEC上达50.99的F0.5和57.17的精确率,优于以往方法
- 适合需要高精度、低冗余纠错的应用场景
基于大语言模型的中文语法纠错系统面临两大挑战:通用模型缺乏细微语法差异的领域先验知识,且基于最大似然估计的监督微调无法优化以精度为导向的指标,导致系统性过度修正。本文提出CSRP,一种三阶段框架:首先在590万条平衡样本上进行持续预训练以内化领域知识;其次采用显式错误推理的思维链微调,提升诊断可解释性;最后通过组相对策略优化与新型效率感知奖励,明确惩罚不必要的修改。在NACGEC基准测试中,CSRP取得50.99的F0.5和57.17的精确率,显著超越先前最优结果,有效缓解了MLE训练模型的过度修正偏差。该方法还将拼写纠错性能提升至59.61 F1,超过GPT-4达5.20点。消融实验表明,强化学习对齐阶段相较微调基线带来8%相对增益,且该收益与大规模持续预训练独立,验证了显式优化编辑效率对高质量纠错的关键作用。代码已开源。
原文摘要 · Abstract (English)
Large Language Model (LLM) based Chinese Grammatical Error Correction (CGEC) systems face two critical challenges: general-purpose models lack specialized linguistic priors for subtle grammatical distinctions, and Supervised Fine-Tuning (SFT) with Maximum Likelihood Estimation fails to optimize for precision-focused metrics, leading to systematic over-correction. We propose CSRP, a three-stage framework that progressively builds correction capability through Continual Pre-training (CPT) on 5.9M balanced samples to internalize domain knowledge, Chain-of-Thought SFT with explicit error reasoning for diagnostic transparency, and Group Relative Policy Optimization with a novel Efficiency-Aware Reward that explicitly penalizes unnecessary edits. On the NACGEC benchmark, CSRP achieves state-of-the-art performance with 50.99 $F_{0.5}$ and 57.17 precision, substantially outperforming previous best results while effectively mitigating the over-correction bias inherent in MLE-trained models. Our method also advances CSCD spelling correction to 59.61 F1, surpassing GPT-4 by 5.20 points. Comprehensive ablation studies demonstrate that the RL alignment stage contributes a 8\% relative gain over the SFT baseline, and that this gain is orthogonal to the contribution of large-scale CPT, validating that explicit optimization for edit efficiency is essential for high-quality grammatical error correction. Our code is available at https://github.com/TW-NLP/ChineseErrorCorrector.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。