用大模型翻译数据,让罗马尼亚语关系抽取效果接近英语。
Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian
- 用大模型自动翻译英文数据集到罗马尼亚语,构建跨语言训练集。
- 微调后罗马尼亚语宏F1提升超22个百分点,跨语言差距缩小至1.4个百分点。
- 小模型如罗马尼亚语BERT表现接近大模型,适合计算资源有限场景。
低资源语言的关系抽取受限于标注语料匮乏。本文通过大模型驱动的翻译流程,将SemEval-2010 Task 8英文基准数据集翻译为罗马尼亚语,评估Gemma 4 31B在零样本、少样本及QLoRA微调下的表现,并对比四种编码器基线(参数量125M至560M):XLM-RoBERTa(base与large)、罗马尼亚语BERT、RoBERT-large。采用两种任务范式:带标记实体的关系分类与端到端抽取。结果表明,在仅提示(prompt-only)设置下,罗马尼亚语性能较英语下降3至5个百分点;少样本提示带来的增益有限;而QLoRA微调使双语言宏F1提升超过22个百分点,跨语言差距由3.3降至1.4个百分点。尽管参数量仅为50至250倍小,编码器基线在罗马尼亚语上仅落后1至4个百分点,其中125M的罗马尼亚语BERT表现媲美278M的多语言XLM-R。因此,在计算资源敏感的部署场景中,使用31B规模模型进行单任务关系抽取的必要性较弱。研究已公开翻译后的数据集、评估代码与训练模型。
原文摘要 · Abstract (English)
Relation extraction (RE) for low-resource languages is typically constrained by the lack of annotated corpora. We investigate the feasibility of cross-lingual RE for Romanian by combining automatic dataset translation with large language model (LLM) inference. We translate the SemEval-2010 Task 8 benchmark from English to Romanian using an LLM-based translation pipeline and evaluate Gemma 4 31B under zero-shot, few-shot, and QLoRA fine-tuned configurations, against four encoder baselines spanning 125M to 560M parameters: XLM- RoBERTa (base and large), Romanian BERT, and RoBERT- large. We assess two task formulations: relation classification with marked entities and end-to-end extraction. Our results show that Romanian incurs a 3 to 5 percentage point (pp) drop relative to English in prompt-only settings, that few-shot prompting provides marginal gains over zero-shot, and that QLoRA fine-tuning improves macro F1-Score by more than 22 percentage points in both languages while reducing the cross-lingual gap from 3.3 to 1.4pp. The encoder baselines come within 1-4pp of QLoRA Gemma on Romanian despite being 50-250 times smaller, with monolingual Romanian BERT at 125M parameters matching multilingual XLM-R at 278M. The case for using a 31B model for single-task RE on Romanian is therefore weak in deployment scenarios where compute matters. We release the translated dataset, evaluation code, and trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。