首个针对低资源语言的拼写纠错实证研究，发现大模型在数据充足时表现更优。
LMSpell: Neural Spell Checking for Low-Resource Languages
- 对比多种预训练模型在拼写纠错中的表现，首次覆盖低资源语言
- 大模型在大规模微调数据下超越编码器和编码器-解码器模型
- 开源工具LMSpell支持多模型评估，适合低资源语言研究者使用
拼写纠错对低资源语言（LRLs）仍是难题。尽管预训练语言模型（PLMs）已被用于拼写纠错，但其应用仍局限于少数语言，且缺乏跨模型的系统比较。本文首次对PLMs在拼写纠错中的有效性进行实证研究，涵盖低资源语言。结果表明，当微调数据量大时，大型语言模型（LLMs）优于编码器型和编码器-解码器型模型，这一现象即使在未在该语言上预训练的LLM中也成立。我们发布LMSpell，一个易于使用的跨PLM拼写纠错工具包，包含可缓解大模型幻觉的评估函数。此外，通过僧伽罗语案例研究，揭示了低资源语言拼写纠错的困境。
原文摘要 · Abstract (English)
Spell correction is still a challenging problem for low-resource languages (LRLs). While pretrained language models (PLMs) have been employed for spell correction, their use is still limited to a handful of languages, and there has been no proper comparison across PLMs. We present the first empirical study on the effectiveness of PLMs for spell correction, which includes LRLs. We find that Large Language Models (LLMs) outperform their counterparts (encoder-based and encoder-decoder) when the fine-tuning dataset is large. This observation holds even in languages for which the LLM is not pre-trained. We release LMSpell, an easy- to use spell correction toolkit across PLMs. It includes an evaluation function that compensates for the hallucination of LLMs. Further, we present a case study with Sinhala to shed light on the plight of spell correction for LRLs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。