首个希腊金融大模型与评测基准,解决希腊语金融NLP数据稀缺问题。
Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance
- 构建首个希腊金融专用评测基准Plutus-ben和80亿参数模型Plutus-8B。
- 在5类任务上评估22个模型,发现跨语言迁移效果有限且需专业金融知识。
- 适合研究多语言金融NLP、希腊语自然语言处理或低资源语言建模的学者。
尽管希腊在全球经济中具有重要地位,但因希腊语语言复杂性和领域数据稀缺,大语言模型(LLMs)在希腊金融语境下的应用仍鲜受关注。以往多语言金融自然语言处理研究揭示了显著性能差异,但此前缺乏专门针对希腊语的金融评测基准或专属模型。为此,我们提出Plutus-ben——首个希腊金融评估基准,以及Plutus-8B——首个经希腊领域数据微调的希腊金融大语言模型。Plutus-ben涵盖五项核心金融自然语言处理任务:数值与文本命名实体识别、问答、抽象摘要生成和主题分类,支持系统化、可复现的模型评估。为支撑这些任务,我们构建了三个由母语专家标注的高质量希腊金融数据集,并融合两个现有资源。对22个大模型在Plutus-ben上的全面评估表明,希腊金融NLP仍具挑战性,主要源于语言复杂性、领域术语及金融推理能力不足。结果凸显了跨语言迁移的局限性、训练模型需具备金融专业知识的重要性,以及将金融大模型适配希腊语文本的困难。我们已公开发布Plutus-ben、Plutus-8B及所有相关数据集,以促进可复现研究,推动希腊金融NLP发展,实现金融领域更广泛的多语言包容性。
原文摘要 · Abstract (English)
Despite Greece's pivotal role in the global economy, large language models (LLMs) remain underexplored for Greek financial context due to the linguistic complexity of Greek and the scarcity of domain-specific datasets. Previous efforts in multilingual financial natural language processing (NLP) have exposed considerable performance disparities, yet no dedicated Greek financial benchmarks or Greek-specific financial LLMs have been developed until now. To bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation Benchmark, and Plutus-8B, the pioneering Greek Financial LLM, fine-tuned with Greek domain-specific data. Plutus-ben addresses five core financial NLP tasks in Greek: numeric and textual named entity recognition, question answering, abstractive summarization, and topic classification, thereby facilitating systematic and reproducible LLM assessments. To underpin these tasks, we present three novel, high-quality Greek financial datasets, thoroughly annotated by expert native Greek speakers, augmented by two existing resources. Our comprehensive evaluation of 22 LLMs on Plutus-ben reveals that Greek financial NLP remains challenging due to linguistic complexity, domain-specific terminology, and financial reasoning gaps. These findings underscore the limitations of cross-lingual transfer, the necessity for financial expertise in Greek-trained models, and the challenges of adapting financial LLMs to Greek text. We release Plutus-ben, Plutus-8B, and all associated datasets publicly to promote reproducible research and advance Greek financial NLP, fostering broader multilingual inclusivity in finance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。