首个面向斯洛伐克语的通用语言理解基准，覆盖九类任务
skLEP: A Slovak General Language Understanding Benchmark
- 构建涵盖词级、句对、文档级的九项任务，专为斯洛伐克语设计
- 首次系统评估多种斯洛伐克语及多语言预训练模型在该基准上的表现
- 开源数据集、工具包与排行榜，推动斯洛伐克语自然语言研究
本文提出 skLEP，首个专为评估斯洛伐克语自然语言理解（NLU）模型而设计的综合性基准。该基准包含九项涵盖词级、句对级和文档级挑战的多样化任务，全面评估模型能力。为构建此基准，我们创建了专属于斯洛伐克语的新原始数据集，并精心翻译了现有的英文 NLU 资源。本文还首次系统性地评估了多种斯洛伐克语专用、多语言及英文预训练语言模型在 skLEP 任务上的表现。最后，我们公开发布完整基准数据、一个支持模型微调与评估的开源工具包，以及位于 https://github.com/slovak-nlp/sklep 的公共排行榜，旨在促进可复现性并推动未来斯洛伐克语 NLU 研究。
原文摘要 · Abstract (English)
In this work, we introduce skLEP, the first comprehensive benchmark specifically designed for evaluating Slovak natural language understanding (NLU) models. We have compiled skLEP to encompass nine diverse tasks that span token-level, sentence-pair, and document-level challenges, thereby offering a thorough assessment of model capabilities. To create this benchmark, we curated new, original datasets tailored for Slovak and meticulously translated established English NLU resources. Within this paper, we also present the first systematic and extensive evaluation of a wide array of Slovak-specific, multilingual, and English pre-trained language models using the skLEP tasks. Finally, we also release the complete benchmark data, an open-source toolkit facilitating both fine-tuning and evaluation of models, and a public leaderboard at https://github.com/slovak-nlp/sklep in the hopes of fostering reproducibility and drive future research in Slovak NLU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。