Fietje是专为荷兰语设计的小型开源语言模型，性能媲美更大模型。
Fietje: An open, efficient LLM for Dutch
- 基于Phi 2微调，专为荷兰语优化的小型模型
- 在多个任务上表现超越更早的大型荷兰语模型
- 完全开源，适合研究者和开发者复现与改进
本文介绍Fietje，一个专为荷兰语设计的小型语言模型系列。该模型基于27亿参数的英语中心模型Phi 2，发布时即展现出与更大模型相当的竞争力。本工作强调透明性与可复现性：Fietje完全开源，模型权重、数据集、训练与评估代码均公开可获取。论文在涵盖推理、情感分析、常识知识、语法可接受性及词义消歧的广泛评测基准上，评估了Fietje及其他多个模型的表现。结果表明，近期小型模型在性能上已超越此前为荷兰语微调的旧版大型模型，预示荷兰语处理领域快速发展。未来持续优化将使模型能力进一步提升，应用范围更广。Fietje仅是提升荷兰语技术可及性的阶段性成果。
原文摘要 · Abstract (English)
This paper introduces Fietje, a family of small language models (SLMs) specifically designed for the Dutch language. The model is based on Phi 2, an English-centric model of 2.7 billion parameters. Fietje demonstrated competitive results with larger language models upon its release. A core emphasis of this work is transparency and reproducibility: Fietje is fully open-source, with model weights, datasets, training, and evaluation code all publicly accessible. The paper discusses the performance of Fietje and many other models on an extensive evaluation suite of benchmarks on reasoning, sentiment analysis, world knowledge, linguistic acceptability and word sense disambiguation. Evaluation results illustrate the rapid progress in the field of LLMs, where recent small models outperform older, larger models that were fine-tuned for Dutch. This trend signals an exciting future for Dutch language processing, suggesting that even compact LLMs are becoming increasingly capable. Furthermore, ongoing and future efforts to adapt LLMs to Dutch are poised to enhance these models even further, broadening their applicability and accessibility. Fietje is only an intermediate step in improving accessibility to language technology for users of the Dutch language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。