用4比特量化和LoRA微调Llama-3.1，让模型高效回答约旦法律问题。
Parameter Efficient Fine Tuning Llama 3.1 for Answering Arabic Legal Questions: A Case Study on Jordanian Laws
- 采用LoRA适配器与4比特量化，实现资源高效微调。
- 在6000对法律问答数据上训练，性能优于基线模型。
- 适合需要低成本部署阿拉伯语法律AI的研究者与开发者。
本研究以约旦法律为案例，探索Llama-3.1大语言模型在阿拉伯语问答任务中的微调。使用两个版本的模型——Llama-3.1-8B-bnb-4bit 和 Llama-3.1-8B-Instruct-bnb-4bit——通过参数高效微调（PEFT）结合LoRA适配器，并在Unsloth框架下进行加速训练。基于约旦法律构建了包含6000个问答对的定制数据集，采用结构化提示格式。通过BLEU和ROUGE指标评估微调后模型与原始基线模型的性能。结果表明，模型在法律推理与准确性方面均有提升，同时借助量化与优化微调策略实现了资源效率。该工作验证了大模型在阿拉伯语法律领域的适应潜力，展示了高效微调技术在特定领域任务中的应用价值。
原文摘要 · Abstract (English)
This study uses Jordanian law as a case study to explore the fine-tuning of the Llama-3.1 large language model for Arabic question-answering. Two versions of the model - Llama-3.1-8B-bnb-4bit and Llama-3.1-8B-Instruct-bnb-4bit - were fine-tuned using parameter-efficient fine-tuning (PEFT) with LoRA adapters and 4-bit quantized models, leveraging the Unsloth framework for accelerated and resource-efficient training. A custom dataset of 6000 legal question-answer pairs was curated from Jordanian laws and formatted into structured prompts. Performance was evaluated using the BLEU and the ROUGE metrics to compare the fine-tuned models to their respective base versions. Results demonstrated improved legal reasoning and accuracy while achieving resource efficiency through quantization and optimized fine-tuning strategies. This work underscores the potential of adapting large language models for Arabic legal domains and highlights effective techniques for fine-tuning domain-specific tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。