用提示词让大模型实现乌克兰语语法纠错，效果接近微调模型。
How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?
- 采用乌克兰语最小修改提示，精准表达语言规则
- 最佳方案达F0.5=69.22，接近微调模型90%性能
- 适合想低成本部署乌克兰语纠错系统的开发者
微调的大语言模型在乌克兰语语法纠错（GEC）中占主导地位，而通过API访问的通用大模型在最小修改基准上仍几乎未被测试。我们评估了来自四大厂商的11个商用大模型及一个开源乌克兰语模型，在UNLP 2023 GEC-only基准上的表现，比较零样本、少样本、最小修改以及大模型辅助提示优化策略。最优配置（Gemini 3.1-Pro）达到F0.5=69.22，缩小了超过90%与微调最先进模型（F0.5=73.14）的差距。零样本提示中，仅Claude系列模型从乌克兰语指令中获益；所有模型的最佳结果均使用乌克兰语最小修改提示，其语言特异性要求用乌克兰语精确表达。在最小修改+少样本基础上进行大模型辅助提示优化，取得最高分。详细最小修改指令对标点和大小写错误提升最大，但导致部分低频错误类别被忽略。误差分析揭示五种与乌克兰语特有语言现象相关的重复误改模式。代码、提示和输出已公开。
原文摘要 · Abstract (English)
Fine-tuned Large Language Models (LLMs) dominate in Ukrainian grammatical error correction (GEC), while API-accessed LLMs remain nearly untested on minimal-edit benchmarks. We evaluate 11 commercial LLMs from four providers and one open-source Ukrainian model on the UNLP 2023 GEC-only benchmark, comparing zero-shot, few-shot, minimal-edits, and LLM-assisted prompt optimization strategies. Our best configuration (Gemini 3.1-Pro) reaches F0.5=69.22, closing over 90% of the gap to fine-tuned SOTA (F0.5=73.14). For zero-shot prompts, only Claude models benefit from Ukrainian instructions. However, the best overall results for all models use Ukrainian minimal-edits prompts, whose language-specific rules require Ukrainian to express precisely. LLM-assisted prompt optimization on top of minimal-edits + few-shot achieves the highest score. Detailed minimal-edits instructions yield the largest gains for punctuation and case errors but cause the model to abandon several low-frequency categories. Delving into error analysis, we identify five recurring overcorrection patterns tied to Ukrainian-specific linguistic phenomena. Code, prompts, and outputs are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。