首个面向欧洲葡萄牙语的开源大模型评估榜单，涵盖语言、文化与伦理表现。
CLARIN-PT-LDB: An Open LLM Leaderboard for Portuguese to assess Language, Culture and Civility
- 构建专用于欧洲葡萄牙语的大模型评估榜单，填补领域空白。
- 新增针对文化契合度与模型安全性的评测基准，提升评估维度。
- 适合关注葡语AI生态、本土化模型评估的研究者与开发者。
本文介绍了专为欧洲葡萄牙语（PT-PT）设计的开源大语言模型（LLM）评估榜单及其配套基准测试。该榜单旨在弥补当前欧洲葡萄牙语大模型缺乏专用评估体系的短板。研究引入了多项新基准，特别关注以往未被覆盖的方面，如模型的安全防护机制及对葡萄牙文化的适配性。评估榜单已上线 Hugging Face 空间：https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard，支持持续更新与社区参与。
原文摘要 · Abstract (English)
This paper reports on the development of a leaderboard of Open Large Language Models (LLM) for European Portuguese (PT-PT), and on its associated benchmarks. This leaderboard comes as a way to address a gap in the evaluation of LLM for European Portuguese, which so far had no leaderboard dedicated to this variant of the language. The paper also reports on novel benchmarks, including some that address aspects of performance that so far have not been available in benchmarks for European Portuguese, namely model safeguards and alignment to Portuguese culture. The leaderboard is available at https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。