探索高效识别阿拉伯方言的模型方法，发现参数高效微调更优。
Exploring Data and Parameter Efficient Strategies for Arabic Dialect Identifications
- 采用软提示和LoRA等参数高效技术提升模型性能
- LoRA微调在多数据集上超越全量微调效果
- 大模型在零样本/少样本下辨识方言能力较弱
本文探讨了用于阿拉伯方言识别（ADI）的不同数据高效与参数高效方法。研究对比了前缀调优、提示调优、P-tuning及P-tuning V2等软提示策略，以及LoRA重参数化方法。在数据高效方面，分析了零样本与少样本提示下的表现；在参数高效方面，在多个主流数据集上使用阿拉伯语专用编码器模型进行实验，并评估了开源解码器模型、通用多语言模型Phi-3.5及阿拉伯语专用模型SILMA在n-shot设置下的表现。结果表明，大语言模型在零样本或少样本场景下难以区分方言细微差异，而软提示编码器表现优于原始模型，但基于LoRA的微调模型性能最佳，甚至超过全量微调。
原文摘要 · Abstract (English)
This paper discusses our exploration of different data-efficient and parameter-efficient approaches to Arabic Dialect Identification (ADI). In particular, we investigate various soft-prompting strategies, including prefix-tuning, prompt-tuning, P-tuning, and P-tuning V2, as well as LoRA reparameterizations. For the data-efficient strategy, we analyze hard prompting with zero-shot and few-shot inferences to analyze the dialect identification capabilities of Large Language Models (LLMs). For the parameter-efficient PEFT approaches, we conducted our experiments using Arabic-specific encoder models on several major datasets. We also analyzed the n-shot inferences on open-source decoder-only models, a general multilingual model (Phi-3.5), and an Arabic-specific one(SILMA). We observed that the LLMs generally struggle to differentiate the dialectal nuances in the few-shot or zero-shot setups. The soft-prompted encoder variants perform better, while the LoRA-based fine-tuned models perform best, even surpassing full fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。