用数据增强提升大模型对阿拉伯文化知识的表示能力
CultranAI at PalmX 2025: Data Augmentation for Cultural Knowledge Representation
- 构建22000+文化相关选择题数据集，融合Palm与PalmX数据
- Fanar-1-9B-Instruct模型在测试集上达70.5%准确率，开发集84.1%
- 适合关注多语言文化知识表征与小样本学习的研究者
本文报告了我们在PalmX文化评估共享任务中的参与情况。我们的系统CultranAI聚焦于大语言模型（LLMs）在阿拉伯文化知识表示中的数据增强与LoRA微调。我们对比了多个LLM以确定最优模型。除使用PalmX数据集外，还通过引入Palm数据集并人工构建了一个包含超过22,000个基于文化情境的选择题数据集。实验表明，Fanar-1-9B-Instruct模型表现最佳。我们在22,000+道增强题目组成的联合数据集上对该模型进行微调。在盲测集上，系统排名第五，准确率为70.50%；在PalmX开发集上准确率达84.1%。
原文摘要 · Abstract (English)
In this paper, we report our participation to the PalmX cultural evaluation shared task. Our system, CultranAI, focused on data augmentation and LoRA fine-tuning of large language models (LLMs) for Arabic cultural knowledge representation. We benchmarked several LLMs to identify the best-performing model for the task. In addition to utilizing the PalmX dataset, we augmented it by incorporating the Palm dataset and curated a new dataset of over 22K culturally grounded multiple-choice questions (MCQs). Our experiments showed that the Fanar-1-9B-Instruct model achieved the highest performance. We fine-tuned this model on the combined augmented dataset of 22K+ MCQs. On the blind test set, our submitted system ranked 5th with an accuracy of 70.50%, while on the PalmX development set, it achieved an accuracy of 84.1%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。