评测大模型在罗马尼亚语重音恢复上的表现，发现GPT-4o效果最好。
Evaluating Large Language Models for Diacritic Restoration in Romanian Texts: A Comparative Study
- 对比多种大模型在罗马尼亚语重音恢复任务中的表现。
- GPT-4o准确率最高，远超基线，而Llama系列波动较大。
- 适合研究低资源语言NLP或重音恢复的开发者参考。
自动重音恢复对拥有丰富重音符号的语言（如罗马尼亚语）的文本处理至关重要。本研究评估了多种大语言模型（LLMs）在罗马尼亚语重音恢复任务中的表现。使用一个全面的语料库，测试了包括OpenAI的GPT-3.5、GPT-4、GPT-4o，Google的Gemini 1.0 Pro，Meta的Llama 2和Llama 3，MistralAI的Mixtral 8x7B Instruct，airoboros 70B，以及OpenLLM-Ro的RoLlama 2 7B在内的多个模型，采用从零样本到复杂多步指令的多种提示模板。结果显示，GPT-4o等模型表现出高重音恢复准确率，持续优于中性回声基线，而Meta的Llama系列则表现出更大波动性。这些发现凸显了模型架构、训练数据和提示设计对重音恢复性能的影响，并为改进重音丰富语言的NLP工具指明了前景。
原文摘要 · Abstract (English)
Automatic diacritic restoration is crucial for text processing in languages with rich diacritical marks, such as Romanian. This study evaluates the performance of several large language models (LLMs) in restoring diacritics in Romanian texts. Using a comprehensive corpus, we tested models including OpenAI's GPT-3.5, GPT-4, GPT-4o, Google's Gemini 1.0 Pro, Meta's Llama 2 and Llama 3, MistralAI's Mixtral 8x7B Instruct, airoboros 70B, and OpenLLM-Ro's RoLlama 2 7B, under multiple prompt templates ranging from zero-shot to complex multi-shot instructions. Results show that models such as GPT-4o achieve high diacritic restoration accuracy, consistently surpassing a neutral echo baseline, while others, including Meta's Llama family, exhibit wider variability. These findings highlight the impact of model architecture, training data, and prompt design on diacritic restoration performance and outline promising directions for improving NLP tools for diacritic-rich languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。