通过捕捉模型内部错误表示，提升多语言语法纠错的少样本性能。
Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction
- 从大模型内部状态提取语法错误编码GER，实现语义中立的错误表征。
- 在高资源语言上达到与闭源模型相当的精度，在低资源语言上提升最多20%。
- 适用于追求可解释性与资源效率的多语言语法纠错研究。
语法纠错（GEC）旨在检测并修正语法使用错误。尽管具备上下文学习能力的大语言模型（LLMs）在多个自然语言处理任务中取得显著进展，其在少样本条件下的GEC表现仍不理想。这主要源于难以检索到能捕捉错误模式而非语义相似性的上下文示例。本文证明，大模型可通过其内部状态固有地捕获语法错误信息。我们从中提取出语法错误表征（GER），一种具有信息量且语义中立的错误编码。基于GER的新检索方法在多语言GEC数据集的上下文学习设置下显著提升性能，提高纠正精度。在高资源语言上，80亿参数开源模型的表现达到Deepseek2.5和GPT-4o-mini等闭源模型水平；在低资源语言上，$F_{0.5}$得分相较基线最高提升1.20倍。该方法为多语言GEC提供了更精确且资源高效的解决方案，为可解释性研究开辟新方向。
原文摘要 · Abstract (English)
Grammatical Error Correction (GEC) involves detecting and correcting the wrong usage of grammar. While large language models (LLMs) with in-context learning (ICL) capabilities have shown significant progress on various natural language processing (NLP) tasks, their few-shot performance on GEC remains suboptimal. This is mainly due to the challenge of retrieving suitable in-context demonstrations that capture error patterns instead of semantic similarity. In this paper, we demonstrate that LLMs can inherently capture information related to grammatical errors through their internal states. From these states, we extract the Grammatical Error Representation (GER), an informative and semantically neutral encoding of grammatical errors. Our novel GER-based retrieval method significantly boosts performance in ICL settings on multilingual GEC datasets, improving the precision of correction. For high-resource languages, our results on 8B-sized open-source models match those of closed-source models such as Deepseek2.5 and GPT-4o-mini. For low-resource languages, our $F_{0.5}$ scores surpass the baseline by up to a factor of 1.20. This method provides a more precise and resource-efficient solution for multilingual GEC, offering a promising direction for interpretable GEC research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。