用小模型在越南语阅读理解上超越大模型，证明高效微调可行
Investigating Recent Large Language Models for Vietnamese Machine Reading Comprehension
- 用QLoRA量化微调Llama 3和Gemma模型
- 小模型在越南语数据集上超越BERT和大模型
- 适合低资源语言NLP研究与轻量部署
大型语言模型（LLMs）在机器阅读理解任务中表现优异，但在低资源语言如越南语上的效果仍不明确。本文对两个最先进的大模型——Llama 3（8B参数）和Gemma（7B参数）在越南语阅读理解数据集ViMMRC上进行微调与评估。通过采用量化低秩适配（QLoRA），我们高效完成了微调，并与多个强大的基于LLM的基线模型进行对比。尽管微调后的模型规模小于GPT-3和GPT-3.5，但其性能均优于传统BERT方法及这些更大模型。这表明该微调流程的有效性，展示了现代大模型如何在保持轻量的同时超越旧有模型。通过深入分析，本文探讨了模型性能的多个方面，为低资源语言如越南语的大模型适配提供了重要洞见。研究推动了低资源语言自然语言处理的发展，相关微调模型已公开发布于Hugging Face。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable proficiency in Machine Reading Comprehension (MRC) tasks; however, their effectiveness for low-resource languages like Vietnamese remains largely unexplored. In this paper, we fine-tune and evaluate two state-of-the-art LLMs: Llama 3 (8B parameters) and Gemma (7B parameters), on ViMMRC, a Vietnamese MRC dataset. By utilizing Quantized Low-Rank Adaptation (QLoRA), we efficiently fine-tune these models and compare their performance against powerful LLM-based baselines. Although our fine-tuned models are smaller than GPT-3 and GPT-3.5, they outperform both traditional BERT-based approaches and these larger models. This demonstrates the effectiveness of our fine-tuning process, showcasing how modern LLMs can surpass the capabilities of older models like BERT while still being suitable for deployment in resource-constrained environments. Through intensive analyses, we explore various aspects of model performance, providing valuable insights into adapting LLMs for low-resource languages like Vietnamese. Our study contributes to the advancement of natural language processing in low-resource languages, and we make our fine-tuned models publicly available at: https://huggingface.co/iaiuet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。