通过简单数据增强提升中文错别字纠正效果
EdaCSC: Two Easy Data Augmentation Methods for Chinese Spelling Correction
- 将长句拆分或减少多错字句中的错误进行数据增强
- 在SIGHAN15上达到当前最优性能
- 适合需要改进纠错鲁棒性的实际应用
中文错别字纠正（CSC）旨在检测并修正因发音或字形相似导致的中文句子拼写错误。尽管现有CSC模型融合了拼音或字形特征并取得显著进展，但在处理含多个错字的句子时仍存在困难，且在真实场景下容易过度纠正。与现有以模型为中心的方法不同，本文提出两种数据增强方法：首先将长句拆分为短句，或对含多个错字的句子减少错误数量；随后采用不同训练策略选择最优模型。在SIGHAN基准上的实验表明，该方法优于多数现有模型，在SIGHAN15测试集上实现当前最优性能。
原文摘要 · Abstract (English)
Chinese Spelling Correction (CSC) aims to detect and correct spelling errors in Chinese sentences caused by phonetic or visual similarities. While current CSC models integrate pinyin or glyph features and have shown significant progress,they still face challenges when dealing with sentences containing multiple typos and are susceptible to overcorrection in real-world scenarios. In contrast to existing model-centric approaches, we propose two data augmentation methods to address these limitations. Firstly, we augment the dataset by either splitting long sentences into shorter ones or reducing typos in sentences with multiple typos. Subsequently, we employ different training processes to select the optimal model. Experimental evaluations on the SIGHAN benchmarks demonstrate the superiority of our approach over most existing models, achieving state-of-the-art performance on the SIGHAN15 test set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。