用微调与解码优化，提升大模型对阿拉伯方言的生成能力
Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding
- 采用LoRA微调结合方言平行语料进行适配
- 融合与MBR解码使方言保真度提升且语义准确
- 适合需要精准方言生成的应用场景
大型语言模型（LLMs）正日益支持多语言，涵盖数百种语言，尤其是高资源语言。然而，由于数据有限和语言变体多样，方言仍被严重忽视。本文通过微调预训练模型来提升其在方言上的表现。具体而言，我们在单语及英阿方言平行语料上使用低秩适应（LoRA）进行微调，结合适配器融合与方言感知的最小束宽（MBR）解码，以增强方言忠实度生成与翻译。在叙利亚、摩洛哥和沙特阿拉伯阿拉伯语上的实验表明，融合与MBR可有效提升方言保真度，同时保持语义准确性。该组合提供了一个紧凑且高效的鲁棒阿拉伯方言生成框架。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are becoming increasingly multilingual, supporting hundreds of languages, especially high resource ones. Unfortunately, Dialect variations are still underrepresented due to limited data and linguistic variation. In this work, we adapt a pre-trained LLM to improve dialectal performance. Specifically, we use Low Rank Adaptation (LoRA) fine-tuning on monolingual and English Dialect parallel data, adapter merging and dialect-aware MBR decoding to improve dialectal fidelity generation and translation. Experiments on Syrian, Moroccan, and Saudi Arabic show that merging and MBR improve dialectal fidelity while preserving semantic accuracy. This combination provides a compact and effective framework for robust dialectal Arabic generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。